The core plug-in adds two caches to the wiki store. A global cache and a cache for each tiddler. While the global cache is cleared whenever any tiddler changes, the tiddler cache is only cleared when the associated tiddler changes. The idea of the tiddler cache is, that sometimes we want to calculate information from a tiddler but don't want to recalculate every time we need this information. Imagine a tiddler which contains links to other tiddlers and at one point we want to have a list of the linked tiddlers (LinksFilter). With the tiddler cache we can parse the WikiText, extract the linked tiddlers and put this list in the cache. Until the tiddler is changed, the cache can be used to access this information in the next requests.
The same idea holds for the global cache. For example when a filter string is parsed and a list of matching tiddlers is constructed. This list can be put in the global cache for later reuse until something changes in the store.
Like the Event Mechanism, the cache needs hook functions in the microkernel.
The microkernel calls clearGlobalCache() and clearCache(tiddlertitle) when a tiddler changes.
The core's cache mechanism overwrites this functions.
The functions providing the cache system are added via the wikimethod module type.
TiddlyWiki consists in its heart only of a basic microkernel providing bare tiddlerstore and module system. It is written in JavaScript and suited to run in a browser or as node.js application. The core plug-in extends the microkernel with powerful functions from a central event system to a sophisticated widget system transforming WikiText to dynamic HTML. Because of it's microkernel architecture the application is highly customizable. The plug-in system not only allows to add new modules but also to override existing modules. Most of the components don't refer directly to modules but load them by type, allowing developers to inject additional modules including new saver implementations, widgets or even rules for the WikiText parser. The user interface of TiddlyWiki is written in WikiText and can be customized with the same language, a user normally uses when just writing wiki entries.
A drawback of the core plug-in is it's high complexity. While the microkernel provides just the bare some bare functions and structures, the core plug-in adds a whole bunch of components at once. It can be challenging to decompose the core architecture and understand the connections between the components. This documentation could only cover the most important parts. This gives a developer the choice of building a whole new application on the microkernel or building the application on the core plug-in, including all modules and UI tiddlers.
In conclusion, TiddlyWiki is a interesting piece of software. The focus on tiddlers, the functionality provided in the core and the fact that the core comes with a full blown wiki application puts TiddlyWiki into a personal information management domain, especially when using TiddlyWiki as a single file application storing code and data in a single HTML file. But the highly customizable nature makes TiddlyWiki perfect for this exact domain. A casual user can organize information with tags and metadata and customize the UI to a grade, that he is able to implement and test his own workflows just by using WikiText. JavaScript developers can add whole new features and create completely new single page applications by building on the microkernel or customizing the core plug-in.
During the runtime the data of Tiddlywiki is stored in javascript objects. These objects are synchronized with the DOM-Representation of Tiddlywiki. This means every change of the original data of a Tiddler, fires an event which changes all DOM-Representations of the Tiddler and the javascript object. The barbone Wiki store is created during the boot process and is kept in a object called \$tw.Wiki. This object contains amongst others a hashmap of the different Tiddlers of Tiddlywiki. The Hashmap is used to store the javascript object representation of the different Tiddlers. Furthermore this object is used to manage the tiddlers during runtime, it provides methods for adding tiddlers, search tiddlers by name and delete tiddlers. As shown in the picture below, every change at the DOM triggers an event which changes the corresponding widget which again changes the store of the tiddlers. The whole image shows how WikiText is parsed by a set of rules into the parse tree and this parse tree is rendered as a tree of widgets. This Rendertree is synchronised to the DOM. Every modification on the Rendertree provokes a start of the rendering-pipeline. As well as every change on the wikiText triggers an event at the RenderTree. This Process uses a selective updating so that only the changed parts are updated. This means only widgets which have to change the DOM in consequence of a changed tiddler are refreshed.
The next important part of the application are deserializers. Deserializers are responsible to load tiddlers from various sources. One of the deserializers provided by the microkernel for example can load new tiddlers from the current DOM tree. The counterpart of deserializers are saver. A saver is used to save the complete store or the complete TiddlyWiki for that matter at once. After a user made changes to some tiddlers, this method can be used to download the current state of the store as a completely new TiddlyWiki application.
Another way of persisting data are syncadaptors. Like deserializer and saver, a syncadaptor can load tiddlers into the store and can persist changed tiddlers. But in contrast to deserializer and saver it can load and persist single tiddlers. A syncadaptor can provide a list of loadable tiddlers registers at the store for changes. Now when a tiddler is changed and these changes are written to the store, the syncadaptor is triggered and persists the new changes, for each tiddler individually.
Tiddlers can be persisted from/to harddisk or synced with a remote server.
![]()
TW has two approaches to save the user data. These approaches depends on way you use TW. either you use node.js as a server for TW its saves the tiddlers as plain text in different files or you use TW as standalone in a browser it persists the data within the HTML-File in two Div-Areas depending on whether the encryption of the TiddlyWiki is activated or not. If the TiddlyWiki is not encrypted the data is stored in the Div-Area called "StoreArea". Every created Tiddler is stored in a own Div-area with a few custom values. An example of a saved Tiddler is shown below (\prettyref{lst:data-div}).
<div created="20140611153703343" modified="20140611153734589" tags="testTag" testfield="testvalue" title="TestTiddler" type="text/plain">
<pre>testText</pre>
</div>The Div-Area has the same attributes like the standard tillder fields, listed in (\prettyref{list:TiddlerFields}), all attributes which are not in this list are parsed as a custom field. The only required attribute is the name attribute, all other attributes are optional.\\ With a activated encryption the data is stored in a special Div-Area called "encryptedStoreArea". TiddlyWiki uses the Standford JavaScript Crypto Libary. The encrypted Tiddlers are saved in a JSON string within this Div-Area.
The micro-kernel creates a TiddlyWiki specific data-structure called tiddler. Here you have to separate the different definition of tiddlers. In the architectural view a tiddler is a JavaScript object holding some data. In the overall concept a tiddler is similar to a wiki page in a normal wiki application like wikipedia. In this section we describe the architectural view of a tiddler. The listing below shows the JSON representation of a tiddler object. During the runtime of the TiddlyWiki everything is saved an object like this. Without any plug-in the architecture is not able to persist any kind of data. All the data is stored in a store. This store is a JavaScript object. This store is constructed like a Map with a bunch of key value pairs. Every tiddler has its name as the key in the map and the JavaScript-Object as the value. The tiddler concept is the main data-model within TiddlyWiki, everything from data up to plug-ins is stored as a tiddler.
{"fields":{
"text":"example Text",
"title":"Infrastruktur",
"tags":["vs"],
"modified":"2014-07-01T16:25:01.230Z",
"myField":"myFieldValue",
"created":"2014-07-01T16:22:10.673Z"
}
}Modules with module-type: tiddlerdeserializer can provide functions to create tiddlers out of any textual representation. Each function must be associated with a type like application/json or text/html.
They get the textual representation of the tiddlers, some default field values and the type of the text block as arguments. They return an array of JavaScript objects representing the resulting tiddlers.
Deserializers are not managed by the syncer module. Instead the concept of deserializers is in fact part of the microkernel. This is necessary because the microkernel needs a way of loading tiddlers before it can load the core plug-in and execute it's startup modules. (Due to the fact that the core plug-in and it's modules are represented as tiddlers.)
The load-modules startup module loads additional deserializers and pushes them into the store.
The core plug-in for example contains a few deserializers which can read a whole TiddlyWiki 5 or classic HTML file and load the individual tiddlers into the store.
The most practical way to develop plugins is to use Node.js with the tiddlywiki5 repository to build your plugins, and to use GitHub to manage you files.
First read http://tiddlywiki.com/static/PluginMechanism.html.
Install Git from http://git-scm.com/downloads
Install Node.js from http://nodejs.org/
Hint: GitHub repositories cannot be grouped together into directories, so it is only possible to group by using a naming scheme, e.g. use 'TW5-' as a name prefix with tiddlywiki5 projects to group them together.
Go to https://github.com/ and create new a repository 'pluginname' - choose to add a readme file.
Choose a location in your file system (eg TW5) for you plugin project; issue commands to:
–create the directory–
mkdir TW5
–make a local read-only copy of the tiddlywiki5 repository–
git clone https://github.com/Jermolene/TiddlyWiki5.git TW5
–make a directory for your plugin–
cd TW5
cd plugins
mkdir yourname
cd yourname
mkdir pluginname
–make a local copy of you plugin repository–
git clone https://github.com/yourgithub/pluginname.git pluginname
–go to your files–
cd pluginname
Create the file plugin.info with content:
{
"title": "$:/plugins/yourgithub/pluginname",
"description": "summary of the plugin's purpose",
"author": "yourname",
"version": "0.0.1",
"core-version": ">=5.0.8",
"source": "https://github.com/yourgithub/pluginname",
"plugin-type": "plugin"
}For example files see the plugins in the tiddlywiki5 repository i.e. those located at plugins/tiddlywiki/ - Note in particular that files need to contain information that is used to tell tiddlywiki the name of the tiddler that is to be used in the tiddlywiki in place of the name of the file within the file system.
Modify editions/tw5.com/tiddlywiki.info to include a reference to your plugin directory, i.e. find "plugins": [ and add "yourname/pluginname".
From the TW5 directory issue command
./bin/qbld.shthe resultant file (index.html) will be placed in the build directory, the default build directory is ../jermolene.github.com relative to TW5/
From plugins/yourname/pluginname/ issue commands to:
–add all files–
git add -A–commit to your local repository—
git commit -am "something meaningful about this check in"–copy local changes to github–
git pushMost of the following mechanisms need a way to get notified, when anything in the wiki store changes. The core -plug-in adds an event system to the bare wiki store. The event system provides the ability to listen to events. The most important is the "change" event which notifies the listeners when tiddlers have changed, with a list of the changed tiddlers. The event mechanism is one of the few mechanisms which needs a hook at the microkernel: The microkernel contains an empty function "enqueueTiddlerEvent(event)" and calls this function when a tiddler is added or deleted. The event mechanism from the core plug-in overwrites this function with it's own implementation. The functions providing the event system are added via the wikimethod module type.
The microkernel only contains a bare store and some deserializers to load tiddlers from JSON files or from the DOM of the current HTML file. The core plug-in adds some more deserializers and a new mechanism for persisting and synchronising tiddlers.
This mechanism is provided as a global module in $:/core/modules/syncer.js. The saver module has three responsibilities:
The syncer module is connected mainly to two other modules.
For one it registers to changes at the wiki store (Event Mechanism) and if any changes occur they are synced to the remote store.
Then it provides a function saveWiki(options). This function can be used by other modules. For example the RootWidget uses this function to save the whole wiki or start downloading single tiddlers.
The syncer itself does not provide a concrete implementation of saving, downloading or syncing the tiddlers.
Instead it loads modules of type saver and syncadaptor and manages the saving/syncing process.
Modules with module-type: tiddlerdeserializer can provide functions to create tiddlers out of any textual representation. Each function must be associated with a type like application/json or text/html.
They get the textual representation of the tiddlers, some default field values and the type of the text block as arguments. They return an array of JavaScript objects representing the resulting tiddlers.
Deserializers are not managed by the syncer module. Instead the concept of deserializers is in fact part of the microkernel. This is necessary because the microkernel needs a way of loading tiddlers before it can load the core plug-in and execute it's startup modules. (Due to the fact that the core plug-in and it's modules are represented as tiddlers.)
The load-modules startup module loads additional deserializers and pushes them into the store.
The core plug-in for example contains a few deserializers which can read a whole TiddlyWiki 5 or classic HTML file and load the individual tiddlers into the store.
Modules with module-type: saver provide functionality to save the whole wiki. There are three methods a saver can support:
A saver module has to export two functions. canSave(wiki) returning true if this module is capable of working and create(wiki} returning an instance of a saver object.
This saver object has to provide an info property containing a name, a priority, an array of methods it supports and a method save(text,method,callback). This method is called from TW with the actual text which should be saved, the method which is used and a callback function to report errors: callback("Error while saving") or to notify that saving went well: callback("Saving went well :)"). If the saver method successfully saved the file it has to return true, or false otherwise.
Saves are triggered by messages from the UI. The syncer module uses the saver with the highest priority capable of the requested method to save the file.
The core plug-in contains a saver capable of saving the current state of the wiki to the local hard drive by using a special Firefox extension called Tiddlyfox. If this extension is not available, the savers canSave method would return false. A saver with a lower priority would then ask the user to save the current state as a new HTML file.
A module with module-type: syncadaptor provides functionality to get a list of tiddlers (this list is provided as SkinnyTiddlers, which are normal tiddlers without the text field) and to load, save and delete single tiddlers. A syncadaptor can also provide functions to login and logout so that syncadaptor modules can be used to synchronize tiddlers with a remote server.
The syncer module only uses one syncadaptor and honours a special system tiddler $:/config/SyncFilter containing a filter string. Tiddlers matching this filter string are not synced with a syncadapter.
Most of the following mechanisms need a way to get notified, when anything in the wiki store changes. The core -plug-in adds an event system to the bare wiki store. The event system provides the ability to listen to events. The most important is the "change" event which notifies the listeners when tiddlers have changed, with a list of the changed tiddlers. The event mechanism is one of the few mechanisms which needs a hook at the microkernel: The microkernel contains an empty function "enqueueTiddlerEvent(event)" and calls this function when a tiddler is added or deleted. The event mechanism from the core plug-in overwrites this function with it's own implementation. The functions providing the event system are added via the wikimethod module type.
The core plug-in adds two caches to the wiki store. A global cache and a cache for each tiddler. While the global cache is cleared whenever any tiddler changes, the tiddler cache is only cleared when the associated tiddler changes. The idea of the tiddler cache is, that sometimes we want to calculate information from a tiddler but don't want to recalculate every time we need this information. Imagine a tiddler which contains links to other tiddlers and at one point we want to have a list of the linked tiddlers (LinksFilter). With the tiddler cache we can parse the WikiText, extract the linked tiddlers and put this list in the cache. Until the tiddler is changed, the cache can be used to access this information in the next requests.
The same idea holds for the global cache. For example when a filter string is parsed and a list of matching tiddlers is constructed. This list can be put in the global cache for later reuse until something changes in the store.
Like the Event Mechanism, the cache needs hook functions in the microkernel.
The microkernel calls clearGlobalCache() and clearCache(tiddlertitle) when a tiddler changes.
The core's cache mechanism overwrites this functions.
The functions providing the cache system are added via the wikimethod module type.
The core plug-in introduces a segregation of tiddlers. The tiddler model is central in the whole TiddlyWiki application and can be used in various roles. Because of the fact that a TiddlyWiki user works with tiddlers (taking the role of a wiki page) and tiddlers are the building blocks of the whole application (including modules, UI elements, etc.) we need to identify the internal tiddlers.
The core plug-in introduces the concept of system tiddlers. It builds on the convention that application internal tiddler names start with $:/.
Then the core plug-in introduces a set of new functions to the wiki store which are used to retrieve tiddlers like getTiddlers(options) and forEachTiddler(callback, options).
These functions work with all tiddlers in the store but the options parameter provides the ability to sort tiddlers by a field-name and exclude tiddlers with a specific tag.
By default it doesn't return system tiddlers. To get a list of all tiddlers including system tiddlers, this must be requested explicitly via the options.
If a function wants to present a list of tiddlers to the user it can use this new functions so that internal application tiddlers wouldn't clutter the resulting list.
These functions are added via the wikimethod module type.
The core plug-in extends the store by a simple mechanism to tag a tiddler. This provides the functionality to
{ tag: [tiddler1, tiddler3, tiddler3] }The tags are stored directly in the tiddler. Each tiddler can have a field named "tag", which contains an array of its tags. The above functions use this field to calculate their results and cache them in a global cache.
Organising information with tags is easy, intuitive and is common throughout the web. In most cases the functions mentioned above are not enough and a more sophisticated way of querying is needed. But instead of focusing only on tags TiddlyWiki introduces a querying system that isn't bound to tags or single tiddlers.
This filter mechanism is build on the idea of a pipeline of single filter operators. Filters are noted as strings and can look like
[tag[task]!tag[done]interesting[very]]This example would (implicitly) put all available tiddlers into the pipe.
The first operator tag[task] would only pass tiddlers which are tagged with "task".
The !tag[done] operator is negated and only passes tiddlers which are not tagged with "done".
The last filter operator passes only tiddlers with a field "interesting" set to "very".
So as a result this filter would be used to obtain all tiddlers which are marked as task, aren't already done and are very interesting.
There are many filter operators already build into the core plug-in including the mentioned tag- and field operators, filter operators to sort the tiddlerlist by a specified field, etc.
But more sophisticated operators are possible, too.
An example would be the search-operator. This filter operator looks for the searched string in the text and in the tags of the tiddlers.
If the provided filter operators are not enough, a developer can add new filters by adding a module with the filteroperator type.
Tags and the filter mechanism are used throughout the core plug-in for internal puproses.
Tags which start with $:/ are normally hidden from the casual user, similar to System Tiddlers
The filter mechanism is added to the wiki store with the wikimethod module type.
Welcome to the developer documentation for TiddlyWiki (http://tiddlywiki.com/). It is currently a work in progress as material from two different sources is adapted and merged:
/MyTranslation/TiddlyWiki5)/MyTranslation/jermolene.github.com<repo>/languages for your translation<repo>/core/language/en-GB into your translation folderplugin.info file (see below) in your translation folder<repo>/editions/tw5.com/tiddlywiki.info to add your language to the list./bin/qbld.sh to build TiddlyWiki/MyTranslation/jermolene.github.com/index.html.tid and .multids files in your language folder to translate the English textContent of plugin.info for Joe Bloggs' Welsh translation:
{
"title": "$:/languages/cy-GB",
"name": "cy-GB",
"plugin-type": "language",
"description": "Welsh (British)",
"author": "JoeBloggs",
"core-version": ">=5.0.0"
}MultiTiddlerFiles make it possible to pack the text of several tiddlers in a single text file, simplifying some editing tasks.
Sometimes the master en-GB language tiddlers are updated with revised content or new items. The best way to keep track of language-related commits to TiddlyWiki5:master is to monitor this RSS/Atom feed:
https://github.com/Jermolene/TiddlyWiki5/commits/master/core/language.atom
The recommended technique for building TiddlyWiki plugins involves running TiddlyWiki on Node.js, but there is now an experimental technique for creating plugins directly in the browser.
Loading a plugin in the browser has several consequences:
To make a modified copy of a plugin, one edits the constituent shadow tiddlers (doing this actually overrides the shadow tiddler with a new non-shadow tiddler containing the modified content). The repacking process retrieves the current value of all the shadow tiddlers included in the plugin, and then bundles the new values back into the original plugin tiddler.
Start with a blank TiddlyWiki. It is useful to create a HelloThere tiddler that contains links to various tiddlers that you'll be opening frequently during plugin development:
$:/plugins/yourname/pluginname)$:/plugins/yourname/pluginname/mywidget.js)Click the link to the plugin tiddler to open it. Assuming it doesn't currently exist, it will open with an italicised title, indicating that it is a missing tiddler. Then switch to edit mode and set the following fields on the tiddler:
| Field | Value |
|---|---|
| dependents | Space separated list of dependent plugins (use square brackets for titles containing spaces) |
| description | Plugin description |
| plugin-type | Either "plugin" for a regular plugin, "theme" for a theme, or "language" for a language pack |
| type | Set to "application/json" |
| version | Set to the version number of the plugin (eg "0.0.1") |
Then in the body of the tiddler, insert:
{"tiddlers": {}}Save the plugin tiddler
Create the payload tiddlers by clicking on the links in the HelloThere tiddler from step 1.
Open the browser developer console, and type the following JavaScript statement, but first change the first parameter to the name of your plugin. The second parameter is an optional array of tiddler titles to be added to the plugin:
$tw.utils.repackPlugin("$:/plugins/yourname/pluginname",["$:/plugins/yourname/pluginname/mywidget.js"])You should see a confirmation message, and then if you inspect the plugin tiddler you should see that it has been filled with the payload tiddlers.
Each time you save the plugin the last portion of the version number is automatically incremented. This will ensure that users with an older version of your plugin will be able to install the new version.
Once you've built the plugin for the first time you can omit the second parameter to repackPlugin() unless you are adding a new tiddler:
$tw.utils.repackPlugin("$:/plugins/yourname/pluginname")To remove tiddlers from the plugin specify their titles in the optional third parameter:
$tw.utils.repackPlugin("$:/plugins/yourname/pluginname",null,["$:/plugins/yourname/pluginname/mywidget.js"])Before attempting to repack your plugin you should ensure that the plugin is selected as the current theme or language. Otherwise the shadow tiddlers will not be present.
TiddlyWiki is a personal notebook application based on a wiki application. In addition to a static Web-Site, TiddlyWiki is implemented as a single page application. This is a approach to build rich internet applications, it includes the possibility to put application logic into web-pages to make them dynamically. Furthermore this means the whole application is delivered in one HTML file, consisting of source code to dynamically change the view and behaviour of the application as well as the data of the application. During the runtime nothing must be loaded from a server to the TiddlyWiki application. The HTML file contains everything needed to start the application. TiddlyWiki is highly customisable because of a very sophisticated module concept. Except of a micro-kernel written in JavaScript the whole application consist of a own data-structure called tiddlers and a own markup language called wikiText. Even the modules are realised as tiddlers.
The aim of this documentation is to overview the idea behind the TiddlyWiki application as well as give a overview of the architecture to the reader. This means after reading the documentation the reader is has the knowledge how the overall application works and where the points are where the reader can extend the functionality of the application.
Section Overview:
Macros can be implemented as JavaScript modules as well as via the wikitext syntax.
JavaScript macros are modules with their module-type field set to macro. They must export these three properties:
this points to the widget node invoking the macro.Note that if the params array is missing or blank, then all the supplied parameters are passed to the run() method.
There are several JavaScript macros built into the core which can serve as a jumping off point for your own macros:
https://github.com/Jermolene/TiddlyWiki5/tree/master/core/modules/macros
Note that JavaScript macros work on both the client and the server, and so do not have access to the browser DOM.
Macros are just used to return a chunk of wikitext for further processing. They should not make modifications to tiddlers in the wiki store. The reason is that you cannott control when the macro is called; it may be called repeatedly as part of refresh processing. So it is important that macros do not have any other side effects beyond generating their text.
Messages are events that are triggered by the user. They are generated by widgets for example when the user clicks on a ButtonWidget. Each message has a type property like "tm-delete-tiddler" and a parameter.
{type: "tm-delete-tiddler", param: "MyOldTiddler"}When such a message is created by a widget it sends the message to it's parent widget which sends it to it's own parent widget and so on.
On this way each widget can try to dispatch the message.
This concept is realised in the base widget object.
It provides a function dispatchEvent(message) which is called by the children to bubble the message up the widget tree.
Another function addEventListener(type,listener) can be used to bind a function to a specific message type.
If the listener returns false, the message is send to the parent widget.
The TiddlyWiki core plug-in handles a lot of messages in the NavigatorWidget.
In the universe of TiddlyWiki everything is a tiddler. Even the application logic is stored in tiddlers that are marked as "application/javascript". These tiddlers, which contain application logic, are called modules and a CommonJS compatible module system is responsible for assembling the individual modules into the TiddlyWiki application. The result is a tree representing the whole TiddlyWiki application containing module tiddlers, data tiddlers and some JavaScript functions and objects.
Only a small part of the TiddlyWiki is not managed as tiddlers, the microkernel. The microkernel is the first thing to run, when the application is started and it puts some initial objects and functions into the application tree, which are needed to load and manage tiddlers. After the microkernel built this initial application tree, the remaining parts of the application can be loaded as module tiddlers. Beside some utility functions the most important object that is contributed by the boot kernel is "$tw.wiki", consisting of JavaScript structures and functions that are used to store and manage the loaded tiddlers. Among other things this store can be used to add new tiddlers, remove tiddlers and retrieve tiddlers by name.
The microkernel constructs a initial $tw object containing the needed structures and functions.
The microkernel is responsible for creating a barbone TW environment. It is running under Node.js or in a HTML5 Browser. The Bootkernel just loads enough functionality to load the modules containing the main logic of the application. This boot-kernel contains a few helper methods, the module mechanism as well as the function to create a tiddler and manage them. The boot-kernel also creates the barbone wiki store, which holds all the information of the wiki during the runtime. After creating the store, the boot-kernel is in charge of decrypting the encrypted tiddlers and extracting all the tiddlers e.g. the core module tiddlers embedded in the DOM structure of the HTML file. Furthermore the boot kernel offers the functionality to load tiddlers from a file, when you run TW with Node.js.
This section describes the architecture of the TiddlyWiki-kernel. TiddlyWiki is based on a micro-kernel which provides only a small stack of functions. This design decision was made to introduce a cleaner mechanism for customization of TiddlyWiki. This section also describes the data-model of TiddlyWiki called tiddler. And it gives a overview to the modul system which developers can use to extend the functionality of the TiddlyWiki application.
The TiddlyWiki application is based on a microkernel architecture, that means it separate minimal functional core from the extended functionality. The microkernel provides the functionality to load external extensions to extend its core features. The TiddlyWiki microkernel provides a few helper methods but the main task of the TiddlyWiki kernel is to provide a basic functionality for storing data and loading the extension plug-ins. Within the TiddlyWiki architecture everything is stored as a tiddler. How the architecture of TiddlyWiki stores his data during the runtime of the application is shown in Datamodel, but the kernel provides this datamodel. It also prepares the functionality to store and create these tiddlers. In favour it creates a store to manage the tiddlers during the runtime. Without any extensions the microkernel is not able to persist the tiddlers, this means the data is only holded in the RAM. Here are some example interfaces for working with tiddlers in memory which are provided by the kernel:
An additional feature of the microkernel is the ability to encrypt and decrypt a block of text. To provide this functionality the microkernel has a built in password-vault to store the current used password. The library used to encrypt and decrypt data is the StandfordJavaScript Crypto Libary. This feature allows the micro-kernel to load encrypted tiddlers from the TiddlyWiki file, but it also allows extension plug-ins to use the encrypt functionality e.g. to persist an encrypted TiddlyWiki.
In order to load extension plug-ins the kernel prepares a interface to load and execute these plug-ins. Therefore the micro-kernel provides some deserializers to extract different type of tiddlers e.g. from the TiddlyWiki-File. Within the microkernel a bunch of different deserializer are installed. These deserializer are needed because every tiddler can include a different type of data for example tiddlers can contain javaScript-code, text, html-text or ~JSON-data. Even after packaging a TiddlyWiki application every plug-in is stored as a tiddler within the TiddlyWiki-Document. This feature is specified in the Module System section. Therefore the micro-kernel need the functionality to parse all the different type of tiddlers. To differ between the different type of tiddlers every tiddler has a file type which are generated by the microkernel
The image below shows the startup process of the TiddlyWiki-kernel. The bootprefix is responsible for preparing the kernel to boot on different engines e.g. browsers and node.js. Afterwards the main boot process which includes the microkernel, with the startup-method, is started. After successfully running these steps the main architecture is loaded. The last step is to run the startup modules. These modules are described later. But in brief this is the point where the TiddlyWiki microkernel can be extended by own functionality. Every module marked as "startup" is started after finishing the boot process of the kernel.
The microkernel builds up the essential functions and structures and initiates a startup sequence.
The micro-kernel creates a TiddlyWiki specific data-structure called tiddler. Here you have to separate the different definition of tiddlers. In the architectural view a tiddler is a JavaScript object holding some data. In the overall concept a tiddler is similar to a wiki page in a normal wiki application like wikipedia. In this section we describe the architectural view of a tiddler. The listing below shows the JSON representation of a tiddler object. During the runtime of the TiddlyWiki everything is saved an object like this. Without any plug-in the architecture is not able to persist any kind of data. All the data is stored in a store. This store is a JavaScript object. This store is constructed like a Map with a bunch of key value pairs. Every tiddler has its name as the key in the map and the JavaScript-Object as the value. The tiddler concept is the main data-model within TiddlyWiki, everything from data up to plug-ins is stored as a tiddler.
{"fields":{
"text":"example Text",
"title":"Infrastruktur",
"tags":["vs"],
"modified":"2014-07-01T16:25:01.230Z",
"myField":"myFieldValue",
"created":"2014-07-01T16:22:10.673Z"
}
}After the boot kernel provides the functions used to load tiddlers, the rest of the TiddlyWiki application is loaded as modules.
A module is a tiddler which has the type application/javascript and contains CommonJS compatible JavaScript code. This means a single module provides its public structures and functions in a variable called export. Other modules can obtain these structures and functions by using a global require function.
var Widget = require("$:/core/modules/widgets/widget.js").widget;
// ...
ButtonWidget.prototype = new Widget();In most cases these module tiddlers are packed into a plug-in. Following the "everything is a tiddler" concept, a plug-in is a tiddler, which contains a bunch of other tiddlers. These tiddlers are first converted into a JSON structure which then becomes the body of the plug-in tiddler. This is not restricted to module tiddlers. A plug-in can contain any tiddlers. This way a developer can put for example simple modules, widgets, UI parts written with WikiText, even new filter operators or extensions to the WikiText parser into a plug-in tiddler. In fact the whole TW core is provided as a single plug-in. Tiddlers provided in a plug-in are called shadow tiddlers and can not be edited. Instead, when trying to edit a shadow tiddler, a new tiddler with the same name is created which then "overrides" the shadow tiddler.
Instead of requiring a specific module directly, a module developer can specify the type of the module he is developing by setting the field "module-type" of the containing tiddler. For example, by providing a module-type of "saver", TiddlyWiki knows that this module implements a way of saving the whole wiki and when the user clicks on the save button, TiddlyWiki automaticly considers the provided module to save the current state.
The TiddlyWiki application is based on a microkernel architecture, that means it separate minimal functional core from the extended functionality. The microkernel provides the functionality to load external extensions to extend its core features. The TiddlyWiki microkernel provides a few helper methods but the main task of the TiddlyWiki kernel is to provide a basic functionality for storing data and loading the extension plug-ins. Within the TiddlyWiki architecture everything is stored as a tiddler. How the architecture of TiddlyWiki stores his data during the runtime of the application is shown in Datamodel, but the kernel provides this datamodel. It also prepares the functionality to store and create these tiddlers. In favour it creates a store to manage the tiddlers during the runtime. Without any extensions the microkernel is not able to persist the tiddlers, this means the data is only holded in the RAM. Here are some example interfaces for working with tiddlers in memory which are provided by the kernel:
An additional feature of the microkernel is the ability to encrypt and decrypt a block of text. To provide this functionality the microkernel has a built in password-vault to store the current used password. The library used to encrypt and decrypt data is the StandfordJavaScript Crypto Libary. This feature allows the micro-kernel to load encrypted tiddlers from the TiddlyWiki file, but it also allows extension plug-ins to use the encrypt functionality e.g. to persist an encrypted TiddlyWiki.
In order to load extension plug-ins the kernel prepares a interface to load and execute these plug-ins. Therefore the micro-kernel provides some deserializers to extract different type of tiddlers e.g. from the TiddlyWiki-File. Within the microkernel a bunch of different deserializer are installed. These deserializer are needed because every tiddler can include a different type of data for example tiddlers can contain javaScript-code, text, html-text or ~JSON-data. Even after packaging a TiddlyWiki application every plug-in is stored as a tiddler within the TiddlyWiki-Document. This feature is specified in the Module System section. Therefore the micro-kernel need the functionality to parse all the different type of tiddlers. To differ between the different type of tiddlers every tiddler has a file type which are generated by the microkernel
The image below shows the startup process of the TiddlyWiki-kernel. The bootprefix is responsible for preparing the kernel to boot on different engines e.g. browsers and node.js. Afterwards the main boot process which includes the microkernel, with the startup-method, is started. After successfully running these steps the main architecture is loaded. The last step is to run the startup modules. These modules are described later. But in brief this is the point where the TiddlyWiki microkernel can be extended by own functionality. Every module marked as "startup" is started after finishing the boot process of the kernel.
The microkernel builds up the essential functions and structures and initiates a startup sequence.
The whole application is basically built from three parts. At first, the microkernel provides the basic functionality to handle tiddlers. The second part are tiddlers representing core functionality. These are for example modules which extend the store by more sophisticated functions, UI tiddlers and widget modules, a WikiText parser, sophisticated deserializers, savers, syncadapters, etc. These core modules are provided as plug-in to the microkernel. Consequently, a plug-in is a single tiddler which itself contains multiple tiddlers, forming the plug-in. Each of this tiddler might be a module providing new functionality (i.e. a module tiddler marked with "module-type: saver" can extend the application with new methods of saving the current wiki state.). Tiddlers provided in plug-ins are called shadow tiddlers. They are immutable and can not be edited or deleted but we can create a new tiddler with the same name to override a shadow tiddler.
After the boot kernel provides the functions used to load tiddlers, the rest of the TiddlyWiki application is loaded as modules.
A module is a tiddler which has the type application/javascript and contains CommonJS compatible JavaScript code. This means a single module provides its public structures and functions in a variable called export. Other modules can obtain these structures and functions by using a global require function.
var Widget = require("$:/core/modules/widgets/widget.js").widget;
// ...
ButtonWidget.prototype = new Widget();In most cases these module tiddlers are packed into a plug-in. Following the "everything is a tiddler" concept, a plug-in is a tiddler, which contains a bunch of other tiddlers. These tiddlers are first converted into a JSON structure which then becomes the body of the plug-in tiddler. This is not restricted to module tiddlers. A plug-in can contain any tiddlers. This way a developer can put for example simple modules, widgets, UI parts written with WikiText, even new filter operators or extensions to the WikiText parser into a plug-in tiddler. In fact the whole TW core is provided as a single plug-in. Tiddlers provided in a plug-in are called shadow tiddlers and can not be edited. Instead, when trying to edit a shadow tiddler, a new tiddler with the same name is created which then "overrides" the shadow tiddler.
Instead of requiring a specific module directly, a module developer can specify the type of the module he is developing by setting the field "module-type" of the containing tiddler. For example, by providing a module-type of "saver", TiddlyWiki knows that this module implements a way of saving the whole wiki and when the user clicks on the save button, TiddlyWiki automaticly considers the provided module to save the current state.
{
"name": "tiddlywiki",
"main": "./index.html",
"window": {
"toolbar": true,
"width": 1024,
"height": 768
}
}The first stage of WikiText processing is the parser.
A Parser is provided by a module with module-type: parser and is responsible to transform block of text to a parse-tree.
The parse-tree consists of nested nodes like
{type: "element", tag: <string>, attributes: {}, children: []} - an HTML element
{type: "text", text: <string>} - a text node
{type: "entity", value: <string>} - an HTML entity like © for a copyright symbol
{type: "raw", html: <string>} - raw HTMLThe core plug-in provides a recursive descent WikiText parser which loads it's individual rules from individual modules.
Thus a developer can provide additional rules by using module-type: wikirule. Each rule can produce a list of parse-tree nodes.
A simple example for a wikirule producing a <hr> from --- can be found in horizrule.js
HTML tags can be embedded into WikiText because of the html rule.
This rule matches HTML tag syntax and creates type: "element" nodes.
But the html-rule has another special purpose. By parsing the HTML tag syntax it implicitly parses WikiText widgets.
It the recognises them by the $ character at the beginning of the tag name and instead of producing "element" nodes
it uses the tag name for the type:
{type: "list", tag: "$list", attributes: {}, children: []} - a list elementThe Widgets part will reveal why this makes sense and how each node is transformed into a widget. Another special characteristic of the html-rule or the parse nodes in general is the attributes property. Attributes in the parse-tree are not stored as simple strings but they are nodes of its own to make indirect text references available as attributes as described in Widgets:
{type: "string", value: <string>} - literal string
{type: "indirect", textReference: <textReference>} - indirect through a text referenceTiddlyWiki supports a wide range of methods to persist your data. One of this methods is the HTML5 fallback saver. This methods works on almost every browser. With this method a copy of the entire wiki will be downloaded by the browser. This means you get a new file everytime you hit the save button. To avoid this and because every Browser has a different API to allow writing direct to the file system there a some plugins for the different browsers. These plug-ins allow the user to save direct to the current open TiddlyWiki-File. The Listing below shows the HTML5-compliant to save the changes via the HTML5 fallback saver by downloading the TW as a complete HTML-file.
DownloadSaver.prototype.save = function(text,method,callback) {
...
var link = document.createElement("a");
link.setAttribute("target","_blank");
...
link.setAttribute("href","data:text/html," + encodeURIComponent(text));
...
link.setAttribute("download",filename);
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
return true;
};package.json to the new version numberbin/bld.sh to build the deployment filespackage.json to the previous version numberbin/verbump "5.0.8-beta", substituting the new version numberbin/deploy.shbin/wbld.sh <username> <password>The previous parts of this chapter showed how WikiText is transformed to DOM nodes which dynamically react to tiddler changes and a way to compose tiddlers from other tiddlers. This last part describes how the TiddlyWiki core plug-in starts up a UI build from tiddlers and WikiText.
After the microkernel has loaded it starts executing Startup Modules. The core plug-in contains two startup modules which are responsible to kick off the UI:
rootwidget.js is a startup module and creates an instance of the base widget.
This widget is globally accessible $tw.rootWidget.
The DOM node associated to this widget is the current browser window's DOM (document).
At first, the root widget has no children but provides some basic event handlers (Messages) like:
After the root widget is loaded another startup module $:/core/modules/startup/render.js creates a transclude widget which contains the contents of $:/core/ui/PageTemplate which is now bound to the browsers DOM document. The render function of the transclude widget is initially executed and a listener is registered at the store which executes the refresh function of the transclude widget to trigger the Selective Update process.
Techniques for including other tiddlers and Templates are finally used in $:/core/ui/PageTemplate to build the TiddlyWiki UI only from tiddlers written in WikiText (with widgets implemented in javascript):
For example to implement the list of open wiki pages the $:/core/ui/PageTemplate contains a navigator widget which maintains a list of open tiddlers in a field of $:/StoryList and handles events like tm-navigate by adding a tiddler specified as parameter to the top of the list in $:/StoryList.
The story tiddler transcluded in $:/core/ui/PageTemplate then uses a ListWidget to transclude all tiddlers in $:/StoryList through a special template $:/core/ui/ViewTemplate.
A event of the type tm-close-tiddler would remove a specified tiddler from $:/StoryList.
The Event Mechanism would trigger a changed event which triggers a call of the ListWidget's refresh function which would remove the tiddler from the list, closing the tiddler.
Modules with module-type: saver provide functionality to save the whole wiki. There are three methods a saver can support:
A saver module has to export two functions. canSave(wiki) returning true if this module is capable of working and create(wiki} returning an instance of a saver object.
This saver object has to provide an info property containing a name, a priority, an array of methods it supports and a method save(text,method,callback). This method is called from TW with the actual text which should be saved, the method which is used and a callback function to report errors: callback("Error while saving") or to notify that saving went well: callback("Saving went well :)"). If the saver method successfully saved the file it has to return true, or false otherwise.
Saves are triggered by messages from the UI. The syncer module uses the saver with the highest priority capable of the requested method to save the file.
The core plug-in contains a saver capable of saving the current state of the wiki to the local hard drive by using a special Firefox extension called Tiddlyfox. If this extension is not available, the savers canSave method would return false. A saver with a lower priority would then ask the user to save the current state as a new HTML file.
With Messages a widget is able to put some kind of events into the TiddlyWiki application which is one part of the dynamic behaviour of widgets.
The other part is selective updating.
Widgets are often dependant on tiddler states.
The ListWidget for example can be configured to list all tiddlers which are tagged with "important".
Now, when such a tiddler is changed and it's "important" tag is removed, this change should reflect in the ListWidget.
To allow widgets to react on such changes, each widget can provide a function refresh(changedTiddlers).
The RootWidget is registered to the wiki store, using the Event Mechanism. When an change event occurs it starts to call the refresh function of its children with a list of the changed tiddlers.
Each widget can then decide if it has to change or re-render its DOM representation and call the refresh function of its own children.
This way every time a tiddler or the wiki store itself changes, each widget can instantly react on these changes or ignore them.
Another way of updating are text reference attributes (text references explained in Transclusion and TextReference):
{type: "indirect", textReference: <textReference>}When a widget got a attribute which is a text reference and the refresh function is called, it can check if the text reference references a changed tiddler and update accordingly.
Nearly every state of the UI is stored in tiddlers. A search mechanism for example would use a EditTextWidget which shows an text input field for the search string and a ListWidget to show the results. The EditTextWidget is bound to a tiddler $:/temp/search. Meaning when editing the text in the input field the tiddlers content is changed to this text. On the other hand when the tiddler changes the RootWidget is notified. It then starts calling its childrens refresh methods. Eventually the refresh method of the EditTextWidget is called and it can re-render itself if necessary. This way TiddlyWiki can re-use an already existing data structure which is not only convenient because we don't need to introduce an additional structure but tiddlers managed in the wiki store are already a pretty powerful data structure, supporting an Event Mechanism (so we don't need an additional observer pattern), Caching, Metadata by additional tiddler fields and a way to obtain specific tiddlers.
Modules with module-type: startup have to export a function named startup and may export a name property to identify this module. The startup function will be executed by the boot kernel at the end of the boot process.
// From boot.js:
// Gather up any startup modules
$tw.boot.remainingStartupModules = []; // Array of startup modules
$tw.modules.forEachModuleOfType("startup",function(title,module) {
if(module.startup) {
$tw.boot.remainingStartupModules.push(module);
}
});
// Keep track of the startup tasks that have been executed
$tw.boot.executedStartupModules = Object.create(null);
$tw.boot.disabledStartupModules = $tw.boot.disabledStartupModules || [];
// Repeatedly execute the next eligible task
$tw.boot.executeNextStartupTask();executeNextStartupTask() will execute the remaining startup modules in remainingStartupModules. A startup module can export the variables before and/or after, each containing an array of names of other startup modules. executeNextStartupTask() will use this information to execute the modules in the correct order.
Startup modules can be marked as synchronous by exporting synchronous = true. If synchronous is set to false, the startup function is executed with an callback function as the first argument. The startup function has to call this function to allow subsequent startup modules to get executed. This is necessary when the startup function itself uses asynchronous calls.
A module with module-type: syncadaptor provides functionality to get a list of tiddlers (this list is provided as SkinnyTiddlers, which are normal tiddlers without the text field) and to load, save and delete single tiddlers. A syncadaptor can also provide functions to login and logout so that syncadaptor modules can be used to synchronize tiddlers with a remote server.
The syncer module only uses one syncadaptor and honours a special system tiddler $:/config/SyncFilter containing a filter string. Tiddlers matching this filter string are not synced with a syncadapter.
SyncAdaptorModules encapsulate storage mechanisms that can be used by the SyncMechanism. Two examples are:
fs moduleSyncAdaptorModules are represented as JavaScript tiddlers with the field module-type set to syncadaptor.
The following properties should be exposed via the exports object:
| Property | Description |
|---|---|
| adaptorClass | The JavaScript class for the adaptor |
Nothing should be exported if the adaptor detects that it isn't capable of operating successfully (eg, because it only runs on either the browser or the server, or because a dependency is missing).
Adaptor modules must handle the following methods.
Constructor(options)Initialises a new adaptor instance.
| Parameter | Description |
|---|---|
| options | See below |
Options include:
getTiddlerInfo(tiddler)Gets the supplemental information that the adaptor needs to keep track of for a particular tiddler. For example, the TiddlyWeb adaptor includes a bag field indicating the original bag of the tiddler.
| Parameter | Description |
|---|---|
| tiddler | Target tiddler |
Returns an object storing any additional information required by the adaptor.
getStatus(callback)Retrieves status information from the server. This method is optional.
| Parameter | Description |
|---|---|
| callback | Callback function invoked with parameters err,isLoggedIn,username |
login(username,password,callback)Attempts to login to the server with specified credentials. This method is optional.
| Parameter | Description |
|---|---|
| username | Username |
| password | Password |
| callback | Callback function invoked with parameter err |
logout(callback)Attempts to logout of the server. This method is optional.
| Parameter | Description |
|---|---|
| callback | Callback function invoked with parameter err |
getSkinnyTiddlers(callback)Retrieves a list of skinny tiddlers from the server.
This method is optional. If an adaptor doesn't implement it then synchronisation will be unidirectional from the TiddlyWiki store to the adaptor, but not the other way.
| Parameter | Description |
|---|---|
| callback | Callback function invoked with parameter err,tiddlers, where tiddlers is an array of tiddler field objects |
saveTiddler(tiddler,callback,tiddlerInfo)Saves a tiddler to the server.
| Parameter | Description |
|---|---|
| tiddler | Tiddler to be saved |
| callback | Callback function invoked with parameter err,adaptorInfo,revision |
| tiddlerInfo | The tiddlerInfo maintained by the syncer for this tiddler |
loadTiddler(title,callback)Loads a tiddler from the server.
| Parameter | Description |
|---|---|
| title | Title of tiddler to be retrieved |
| callback | Callback function invoked with parameter err,tiddlerFields |
deleteTiddler(title,callback,tiddlerInfo)Delete a tiddler from the server.
| Parameter | Description |
|---|---|
| title | Title of tiddler to be deleted |
| callback | Callback function invoked with parameter err |
| tiddlerInfo | The tiddlerInfo maintained by the syncer for this tiddler |
The core plug-in introduces a segregation of tiddlers. The tiddler model is central in the whole TiddlyWiki application and can be used in various roles. Because of the fact that a TiddlyWiki user works with tiddlers (taking the role of a wiki page) and tiddlers are the building blocks of the whole application (including modules, UI elements, etc.) we need to identify the internal tiddlers.
The core plug-in introduces the concept of system tiddlers. It builds on the convention that application internal tiddler names start with $:/.
Then the core plug-in introduces a set of new functions to the wiki store which are used to retrieve tiddlers like getTiddlers(options) and forEachTiddler(callback, options).
These functions work with all tiddlers in the store but the options parameter provides the ability to sort tiddlers by a field-name and exclude tiddlers with a specific tag.
By default it doesn't return system tiddlers. To get a list of all tiddlers including system tiddlers, this must be requested explicitly via the options.
If a function wants to present a list of tiddlers to the user it can use this new functions so that internal application tiddlers wouldn't clutter the resulting list.
These functions are added via the wikimethod module type.
The core plug-in extends the store by a simple mechanism to tag a tiddler. This provides the functionality to
{ tag: [tiddler1, tiddler3, tiddler3] }The tags are stored directly in the tiddler. Each tiddler can have a field named "tag", which contains an array of its tags. The above functions use this field to calculate their results and cache them in a global cache.
Organising information with tags is easy, intuitive and is common throughout the web. In most cases the functions mentioned above are not enough and a more sophisticated way of querying is needed. But instead of focusing only on tags TiddlyWiki introduces a querying system that isn't bound to tags or single tiddlers.
This filter mechanism is build on the idea of a pipeline of single filter operators. Filters are noted as strings and can look like
[tag[task]!tag[done]interesting[very]]This example would (implicitly) put all available tiddlers into the pipe.
The first operator tag[task] would only pass tiddlers which are tagged with "task".
The !tag[done] operator is negated and only passes tiddlers which are not tagged with "done".
The last filter operator passes only tiddlers with a field "interesting" set to "very".
So as a result this filter would be used to obtain all tiddlers which are marked as task, aren't already done and are very interesting.
There are many filter operators already build into the core plug-in including the mentioned tag- and field operators, filter operators to sort the tiddlerlist by a specified field, etc.
But more sophisticated operators are possible, too.
An example would be the search-operator. This filter operator looks for the searched string in the text and in the tags of the tiddlers.
If the provided filter operators are not enough, a developer can add new filters by adding a module with the filteroperator type.
Tags and the filter mechanism are used throughout the core plug-in for internal puproses.
Tags which start with $:/ are normally hidden from the casual user, similar to System Tiddlers
The filter mechanism is added to the wiki store with the wikimethod module type.
The boot-kernel is responsible for creating a barbone TW environment. It is running under Node.js or in a HTML5 Browser. The Bootkernel just loads enough functionality to load the modules containing the main logic of the application. This boot-kernel contains a few helper methods, the module mechanism as well as the function to create a tiddler and manage them. The boot-kernel also creates the barbone wiki store, which holds all the information of the wiki during the runtime. After creating the store, the boot-kernel is in charge of decrypting the encrypted tiddlers and extracting all the tiddlers e.g. the core module tiddlers embedded in the DOM structure of the HTML file. Furthermore the boot kernel offers the functionality to load tiddlers from a file, when you run TW with Node.js. All other functionality which is not a part of the boot kernel is added dynamically by modules and plugins. The boot kernel is able to load the core plugins and perform the startup plugins. The core contains the startup modules shown in the picture below.
Beside the boot kernel, TW is completely build from modules. After a short introduction on modules and plug-ins in the context of TW and explaining how they are organized and managed, the following sections will show what type of modules a developer can build and how they hook into the TW architecture. The last section shows the procedure of building an plug-in. This can be used as a tutorial for building your own plug-ins and will show how an advanced user can create solutions for his own use cases only by using the tiddler model and the WikiText markup language.
This section descripes how the data of the wiki is stored within Tiddlywiki during the runtime. And how the complete wiki is persisted.
Following the "anything is a tiddler" concept, even the UI consists of tiddlers. This is possible because tiddlers can not only contain plain text or JavaScript (modules) but they also can contain a special markup text called WikiText. By using WikiText the user can put markup elements like tables or images in a tiddler. To provide some more sophisticated UI elements, WikiText can also contain special widgets like text input fields, checkboxes, dynamic lists etc. In most cases, these widgets are used to directly modify or represent the information contained in other tiddlers. If a tiddler is changed and this change should reflect in an UI element e.g. a widget, a process called selective update takes place. Selective updating means when a tiddler or a set of tiddlers changes, each widget is asked, if changes to this tiddlers would affect its appearance. If so, the respective widget is re-rendered otherwise it remains unchanged.
By managing nearly every part of the application as tiddlers, the application is only needed to provide some basic functionality to manage the individual tiddlers, load and persist them, render them to HTML output and provide a way to register for the changes made to tiddlers. This way the whole wiki application can be build from these simple concepts. Plug-ins can be used to add new functionality to the existing modules or even to replace individual tiddlers/modules, enabling developers to build whole new applications on the TiddlyWiki base system.
A tiddler is the smallest unit of the TiddlyWiki system. It can contain any data like plain text, WikiText markup, JavaScript code (module tiddler), JSON structures (JSON structures might even contain additional tiddlers. Plug-ins are implemented this way to pack multiple tiddlers in a single plug-in tiddler), images in SVG format or even binary images encoded with base64. Internally Tiddlers are immutable objects containing a bunch of key:value pairs called fields. The only required field of a tiddler is the title field. The Standard fields of a tiddler are listed below. Nearly everything in TiddlyWiki is loaded as tiddlers. Plug-ins for example are a bunch of tiddlers that are distributed as a single JSON tiddler. The only exception is the microkernel which isn't a tiddler.
Tiddlers are used in multiple roles and on different levels. A developer uses tiddlers as the basic element containing application code, configuration values and even as a form of variable to save the current UI state.
On a different level, a tiddler is also the basic unit of work for the wiki user, e.g. the individual wiki pages are implemented as tiddlers.
This makes sense for multiple reasons:
Because the UI of TiddlyWiki is build from tiddlers, the wiki user is able to edit the interface of his own TiddlyWiki just by editing a wiki page.
For example to add a list of tiddler links to the sidebar, the user just needs to create a new tiddler, put the links into this tiddler and tag this tiddler with $:/tags/SideBar.
This way the user can customize his work environment just by using mechanisms he already uses to manage his wiki pages.
Tiddlers consist of fields. When using a tiddler as wiki page, the user can use these fields to store meta information, like tags.
Because fields for metadata and especially tags are an easy way for the user to organize his wiki pages, TiddlyWiki provides a special filter mechanism to choose tiddlers using their metadata.
A filter string like [tag[learncard]topic[math]!tag[successful]] would filter all tiddlers tagged with "learncard", with the value "math" in the topic-field and are not tagged with "successful".
A user could use this filter together with the <$list> widget to display a list of all math learncards which are not yet answered successfully in a wiki page.
Another example which shows how the "anything is a tiddler" concept leads to an environment where a single feature brings great benefit is the drag and drop feature. HTML5 standard comes with a native drag and drop feature. TiddlyWiki uses this feature and makes it possible to drag and drop a tiddler from one instance to another. And because anything is a tiddler, this brings the ability to drag and drop individual wiki pages, JavaScript modules, UI components and whole plug-ins between TiddlyWiki instances.
Traditional web applications are bound to ~HTTP-Concepts, including stateless requests to transfer data. In these applications a state is often emulated by the use of sessions, which need to be handled on the client and especially on the server side. These restrictions often lead to a fragmented user experience because the user interface is rebuilt on every data transfer. TiddlyWiki tries to overcome these restrictions and the resulting disadvantages by building on few but basic concepts which loosen the coupling of HTTP and the actual application, eliminating the need of state emulation and resulting in a wiki style single page application with the ability to run in an offline environment.
TiddlyWiki builds on some basic concepts. First, TiddlyWiki should not be perceived as a dynamic web page like traditional server-side generated web pages. Instead TiddlyWiki can be perceived as an application which is written entirely in JavaScript and uses HTML5 and CSS3 to render a GUI. This way TiddlyWiki can be executed in any JavaScript environment like a browser or a node.js instance, while keeping the advantages of a simple application. One of these advantages is, that TiddlyWiki has no need to emulate an application state, as a traditional web application would need to do.
A second idea concerns the storage of the application data. In contrast to a traditional web application, TiddlyWiki doesn't store the data in an external database but simply uses native data structures already existing in JavaScript to store tiddlers, the basic (atomic) element of the TiddlyWiki application, in the memory. Additional core modules provide a way to persist this storage in simple HTML div elements.
Just by building on these simple and basic concepts,
These points already enable TW to be used as an offline-enabled single file web application. Also, by using a server side node.js environment running the same TiddlyWiki application, TiddlyWiki can be used as an online web application. This is realized on server-side by providing an additional module to persist tiddlers into plain text files and on client-side by a module syncing the local data store with the node.js server.
In the universe of TiddlyWiki everything is a tiddler. Even the application logic is stored in tiddlers that are marked as "application/javascript". These tiddlers, which contain application logic, are called modules and a CommonJS compatible module system is responsible for assembling the individual modules into the TiddlyWiki application. The result is a tree representing the whole TiddlyWiki application containing module tiddlers, data tiddlers and some JavaScript functions and objects.
Only a small part of the TiddlyWiki is not managed as tiddlers, the microkernel. The microkernel is the first thing to run, when the application is started and it puts some initial objects and functions into the application tree, which are needed to load and manage tiddlers. After the microkernel built this initial application tree, the remaining parts of the application can be loaded as module tiddlers. Beside some utility functions the most important object that is contributed by the boot kernel is "$tw.wiki", consisting of JavaScript structures and functions that are used to store and manage the loaded tiddlers. Among other things this store can be used to add new tiddlers, remove tiddlers and retrieve tiddlers by name.
The microkernel constructs a initial $tw object containing the needed structures and functions.
The next important part of the application are deserializers. Deserializers are responsible to load tiddlers from various sources. One of the deserializers provided by the microkernel for example can load new tiddlers from the current DOM tree. The counterpart of deserializers are saver. A saver is used to save the complete store or the complete TiddlyWiki for that matter at once. After a user made changes to some tiddlers, this method can be used to download the current state of the store as a completely new TiddlyWiki application.
Another way of persisting data are syncadaptors. Like deserializer and saver, a syncadaptor can load tiddlers into the store and can persist changed tiddlers. But in contrast to deserializer and saver it can load and persist single tiddlers. A syncadaptor can provide a list of loadable tiddlers registers at the store for changes. Now when a tiddler is changed and these changes are written to the store, the syncadaptor is triggered and persists the new changes, for each tiddler individually.
Tiddlers can be persisted from/to harddisk or synced with a remote server.
![]()
Following the "anything is a tiddler" concept, even the UI consists of tiddlers. This is possible because tiddlers can not only contain plain text or JavaScript (modules) but they also can contain a special markup text called WikiText. By using WikiText the user can put markup elements like tables or images in a tiddler. To provide some more sophisticated UI elements, WikiText can also contain special widgets like text input fields, checkboxes, dynamic lists etc. In most cases, these widgets are used to directly modify or represent the information contained in other tiddlers. If a tiddler is changed and this change should reflect in an UI element e.g. a widget, a process called selective update takes place. Selective updating means when a tiddler or a set of tiddlers changes, each widget is asked, if changes to this tiddlers would affect its appearance. If so, the respective widget is re-rendered otherwise it remains unchanged.
The whole application is basically built from three parts. At first, the microkernel provides the basic functionality to handle tiddlers. The second part are tiddlers representing core functionality. These are for example modules which extend the store by more sophisticated functions, UI tiddlers and widget modules, a WikiText parser, sophisticated deserializers, savers, syncadapters, etc. These core modules are provided as plug-in to the microkernel. Consequently, a plug-in is a single tiddler which itself contains multiple tiddlers, forming the plug-in. Each of this tiddler might be a module providing new functionality (i.e. a module tiddler marked with "module-type: saver" can extend the application with new methods of saving the current wiki state.). Tiddlers provided in plug-ins are called shadow tiddlers. They are immutable and can not be edited or deleted but we can create a new tiddler with the same name to override a shadow tiddler.
By managing nearly every part of the application as tiddlers, the application is only needed to provide some basic functionality to manage the individual tiddlers, load and persist them, render them to HTML output and provide a way to register for the changes made to tiddlers. This way the whole wiki application can be build from these simple concepts. Plug-ins can be used to add new functionality to the existing modules or even to replace individual tiddlers/modules, enabling developers to build whole new applications on the TiddlyWiki base system.
A tiddler is the smallest unit of the TiddlyWiki system. It can contain any data like plain text, WikiText markup, JavaScript code (module tiddler), JSON structures (JSON structures might even contain additional tiddlers. Plug-ins are implemented this way to pack multiple tiddlers in a single plug-in tiddler), images in SVG format or even binary images encoded with base64. Internally Tiddlers are immutable objects containing a bunch of key:value pairs called fields. The only required field of a tiddler is the title field. The Standard fields of a tiddler are listed below. Nearly everything in TiddlyWiki is loaded as tiddlers. Plug-ins for example are a bunch of tiddlers that are distributed as a single JSON tiddler. The only exception is the microkernel which isn't a tiddler.
Tiddlers are used in multiple roles and on different levels. A developer uses tiddlers as the basic element containing application code, configuration values and even as a form of variable to save the current UI state.
On a different level, a tiddler is also the basic unit of work for the wiki user, e.g. the individual wiki pages are implemented as tiddlers.
This makes sense for multiple reasons:
Because the UI of TiddlyWiki is build from tiddlers, the wiki user is able to edit the interface of his own TiddlyWiki just by editing a wiki page.
For example to add a list of tiddler links to the sidebar, the user just needs to create a new tiddler, put the links into this tiddler and tag this tiddler with $:/tags/SideBar.
This way the user can customize his work environment just by using mechanisms he already uses to manage his wiki pages.
Tiddlers consist of fields. When using a tiddler as wiki page, the user can use these fields to store meta information, like tags.
Because fields for metadata and especially tags are an easy way for the user to organize his wiki pages, TiddlyWiki provides a special filter mechanism to choose tiddlers using their metadata.
A filter string like [tag[learncard]topic[math]!tag[successful]] would filter all tiddlers tagged with "learncard", with the value "math" in the topic-field and are not tagged with "successful".
A user could use this filter together with the <$list> widget to display a list of all math learncards which are not yet answered successfully in a wiki page.
Another example which shows how the "anything is a tiddler" concept leads to an environment where a single feature brings great benefit is the drag and drop feature. HTML5 standard comes with a native drag and drop feature. TiddlyWiki uses this feature and makes it possible to drag and drop a tiddler from one instance to another. And because anything is a tiddler, this brings the ability to drag and drop individual wiki pages, JavaScript modules, UI components and whole plug-ins between TiddlyWiki instances.
The WikiText is a markup language, created especially for the requirements of the TiddlyWiki application. It is based on Markdown, but extended with some TiddlyWiki specific features. On one hand its a text-to-HTML conversion language and on the other hand its used to provide the interactive features of TiddlyWiki. The aim of this language is to allow the user of the software to focus on the writing. The WikiText is used to format Tiddlers within the TiddlyWiki application. The tags of the WikiText syntax can be used within the standard text input field. During the saving process these tags renders to HTML elements for example:
WikiText:---
Renders as:
HTML:<hr>
WikiText:[img[http://tiddlywiki.com/favicon.ico]]
Renders as: TW
HTML:<img src="http://tiddlywiki.com/favicon.ico">Furthermore the WikiText is used to access the widgets which are integrated in the application.These widgets are used to enhance the the WikiText with a rich functionality. Widgets are based on the ~HTML-Syntax but always starts with a $.
WikiText:
<$button message="tm-close-tiddler">Close Me!</$button> Traditional web applications are bound to ~HTTP-Concepts, including stateless requests to transfer data. In these applications a state is often emulated by the use of sessions, which need to be handled on the client and especially on the server side. These restrictions often lead to a fragmented user experience because the user interface is rebuilt on every data transfer. TiddlyWiki tries to overcome these restrictions and the resulting disadvantages by building on few but basic concepts which loosen the coupling of HTTP and the actual application, eliminating the need of state emulation and resulting in a wiki style single page application with the ability to run in an offline environment.
TiddlyWiki builds on some basic concepts. First, TiddlyWiki should not be perceived as a dynamic web page like traditional server-side generated web pages. Instead TiddlyWiki can be perceived as an application which is written entirely in JavaScript and uses HTML5 and CSS3 to render a GUI. This way TiddlyWiki can be executed in any JavaScript environment like a browser or a node.js instance, while keeping the advantages of a simple application. One of these advantages is, that TiddlyWiki has no need to emulate an application state, as a traditional web application would need to do.
A second idea concerns the storage of the application data. In contrast to a traditional web application, TiddlyWiki doesn't store the data in an external database but simply uses native data structures already existing in JavaScript to store tiddlers, the basic (atomic) element of the TiddlyWiki application, in the memory. Additional core modules provide a way to persist this storage in simple HTML div elements.
Just by building on these simple and basic concepts,
These points already enable TW to be used as an offline-enabled single file web application. Also, by using a server side node.js environment running the same TiddlyWiki application, TiddlyWiki can be used as an online web application. This is realized on server-side by providing an additional module to persist tiddlers into plain text files and on client-side by a module syncing the local data store with the node.js server.
In the universe of TiddlyWiki everything is a tiddler. Even the application logic is stored in tiddlers that are marked as "application/javascript". These tiddlers, which contain application logic, are called modules and a CommonJS compatible module system is responsible for assembling the individual modules into the TiddlyWiki application. The result is a tree representing the whole TiddlyWiki application containing module tiddlers, data tiddlers and some JavaScript functions and objects.
Only a small part of the TiddlyWiki is not managed as tiddlers, the microkernel. The microkernel is the first thing to run, when the application is started and it puts some initial objects and functions into the application tree, which are needed to load and manage tiddlers. After the microkernel built this initial application tree, the remaining parts of the application can be loaded as module tiddlers. Beside some utility functions the most important object that is contributed by the boot kernel is "$tw.wiki", consisting of JavaScript structures and functions that are used to store and manage the loaded tiddlers. Among other things this store can be used to add new tiddlers, remove tiddlers and retrieve tiddlers by name.
The microkernel constructs a initial $tw object containing the needed structures and functions.
The next important part of the application are deserializers. Deserializers are responsible to load tiddlers from various sources. One of the deserializers provided by the microkernel for example can load new tiddlers from the current DOM tree. The counterpart of deserializers are saver. A saver is used to save the complete store or the complete TiddlyWiki for that matter at once. After a user made changes to some tiddlers, this method can be used to download the current state of the store as a completely new TiddlyWiki application.
Another way of persisting data are syncadaptors. Like deserializer and saver, a syncadaptor can load tiddlers into the store and can persist changed tiddlers. But in contrast to deserializer and saver it can load and persist single tiddlers. A syncadaptor can provide a list of loadable tiddlers registers at the store for changes. Now when a tiddler is changed and these changes are written to the store, the syncadaptor is triggered and persists the new changes, for each tiddler individually.
Tiddlers can be persisted from/to harddisk or synced with a remote server.
![]()
Following the "anything is a tiddler" concept, even the UI consists of tiddlers. This is possible because tiddlers can not only contain plain text or JavaScript (modules) but they also can contain a special markup text called WikiText. By using WikiText the user can put markup elements like tables or images in a tiddler. To provide some more sophisticated UI elements, WikiText can also contain special widgets like text input fields, checkboxes, dynamic lists etc. In most cases, these widgets are used to directly modify or represent the information contained in other tiddlers. If a tiddler is changed and this change should reflect in an UI element e.g. a widget, a process called selective update takes place. Selective updating means when a tiddler or a set of tiddlers changes, each widget is asked, if changes to this tiddlers would affect its appearance. If so, the respective widget is re-rendered otherwise it remains unchanged.
The whole application is basically built from three parts. At first, the microkernel provides the basic functionality to handle tiddlers. The second part are tiddlers representing core functionality. These are for example modules which extend the store by more sophisticated functions, UI tiddlers and widget modules, a WikiText parser, sophisticated deserializers, savers, syncadapters, etc. These core modules are provided as plug-in to the microkernel. Consequently, a plug-in is a single tiddler which itself contains multiple tiddlers, forming the plug-in. Each of this tiddler might be a module providing new functionality (i.e. a module tiddler marked with "module-type: saver" can extend the application with new methods of saving the current wiki state.). Tiddlers provided in plug-ins are called shadow tiddlers. They are immutable and can not be edited or deleted but we can create a new tiddler with the same name to override a shadow tiddler.
By managing nearly every part of the application as tiddlers, the application is only needed to provide some basic functionality to manage the individual tiddlers, load and persist them, render them to HTML output and provide a way to register for the changes made to tiddlers. This way the whole wiki application can be build from these simple concepts. Plug-ins can be used to add new functionality to the existing modules or even to replace individual tiddlers/modules, enabling developers to build whole new applications on the TiddlyWiki base system.
A tiddler is the smallest unit of the TiddlyWiki system. It can contain any data like plain text, WikiText markup, JavaScript code (module tiddler), JSON structures (JSON structures might even contain additional tiddlers. Plug-ins are implemented this way to pack multiple tiddlers in a single plug-in tiddler), images in SVG format or even binary images encoded with base64. Internally Tiddlers are immutable objects containing a bunch of key:value pairs called fields. The only required field of a tiddler is the title field. The Standard fields of a tiddler are listed below. Nearly everything in TiddlyWiki is loaded as tiddlers. Plug-ins for example are a bunch of tiddlers that are distributed as a single JSON tiddler. The only exception is the microkernel which isn't a tiddler.
Tiddlers are used in multiple roles and on different levels. A developer uses tiddlers as the basic element containing application code, configuration values and even as a form of variable to save the current UI state.
On a different level, a tiddler is also the basic unit of work for the wiki user, e.g. the individual wiki pages are implemented as tiddlers.
This makes sense for multiple reasons:
Because the UI of TiddlyWiki is build from tiddlers, the wiki user is able to edit the interface of his own TiddlyWiki just by editing a wiki page.
For example to add a list of tiddler links to the sidebar, the user just needs to create a new tiddler, put the links into this tiddler and tag this tiddler with $:/tags/SideBar.
This way the user can customize his work environment just by using mechanisms he already uses to manage his wiki pages.
Tiddlers consist of fields. When using a tiddler as wiki page, the user can use these fields to store meta information, like tags.
Because fields for metadata and especially tags are an easy way for the user to organize his wiki pages, TiddlyWiki provides a special filter mechanism to choose tiddlers using their metadata.
A filter string like [tag[learncard]topic[math]!tag[successful]] would filter all tiddlers tagged with "learncard", with the value "math" in the topic-field and are not tagged with "successful".
A user could use this filter together with the <$list> widget to display a list of all math learncards which are not yet answered successfully in a wiki page.
Another example which shows how the "anything is a tiddler" concept leads to an environment where a single feature brings great benefit is the drag and drop feature. HTML5 standard comes with a native drag and drop feature. TiddlyWiki uses this feature and makes it possible to drag and drop a tiddler from one instance to another. And because anything is a tiddler, this brings the ability to drag and drop individual wiki pages, JavaScript modules, UI components and whole plug-ins between TiddlyWiki instances.
The WikiText is a markup language, created especially for the requirements of the TiddlyWiki application. It is based on Markdown, but extended with some TiddlyWiki specific features. On one hand its a text-to-HTML conversion language and on the other hand its used to provide the interactive features of TiddlyWiki. The aim of this language is to allow the user of the software to focus on the writing. The WikiText is used to format Tiddlers within the TiddlyWiki application. The tags of the WikiText syntax can be used within the standard text input field. During the saving process these tags renders to HTML elements for example:
WikiText:---
Renders as:
HTML:<hr>
WikiText:[img[http://tiddlywiki.com/favicon.ico]]
Renders as: TW
HTML:<img src="http://tiddlywiki.com/favicon.ico">Furthermore the WikiText is used to access the widgets which are integrated in the application.These widgets are used to enhance the the WikiText with a rich functionality. Widgets are based on the ~HTML-Syntax but always starts with a $.
WikiText:
<$button message="tm-close-tiddler">Close Me!</$button> The heart of TiddlyWiki is an extensible representation transformation engine for text and images. Given the text of a tiddler and its associated ContentType, the engine can produce a rendering of the tiddler in a new ContentType. Furthermore, it can efficiently selectively update the rendering to track any changes in the tiddler or its dependents.
The processing pipeline shows how WikiText is parsed by a stack of parse rules into a parse tree. The parse tree is rendered as a tree of widgets, which is synchronised into the DOM via the RefreshMechanism.
DOM events trigger actions on widgets which update the tiddler store. The updates trigger a change event which in turn triggers the refresh mechanism to update the DOM.
When programming TiddlyWiki 5 plugins which make changes outside the scope of the included examples, you will need a more in-depth understanding of TiddlyWiki's internal architecture.
TiddlyWiki's data model is a fairly simple key-value store.
fields member containing members such as title, text, tags and so on.Wiki class defined in boot/boot.js implements simple associative array behaviours like insertion, deletion, iteration, listing keys, and get-by-key.core/modules/wiki.js then extends the Wiki class with functionality such as event dispatch and various cache-backed methods such as getTiddlersWithTag.The active tiddler store can be accessed via $tw.wiki as in this example:
$tw.wiki.makeTiddlerIterator($tw.wiki.getTiddlersWithTag('timeline')
)(function(tiddler, title) {
// Skip templates
if (tiddler.fields.tags.indexOf('templates') >= 0) { return; }
do_something(tiddler);
});Data which should not be visible to end users under normal operation (eg. internal components, plugins, persisted state for GUI widgets) is stored in system tiddlers organised via a set of namespaces.
The similarity between filesystem paths and system tiddler names is intentional and will be used to provide a hierarchical browsing interface in a future TiddlyWiki release.
TiddlyWiki's view layer has a lot in common with desktop widget toolkits and this is the part which is most likely to trip up newcomers. There are two facets to it:
Because TiddlyWiki may re-render content, plugins should treat the DOM as write-only.
In other words, any state you store in the DOM could vanish at any instant and you need to use TiddlyWiki's internal StateMechanism instead.
In a desktop application, the base widget class defines a method such as paint(canvas) which is called in response to expose events or "data has changed" messages.
In TiddlyWiki, the Widget class in core/modules/widgets/widgets.js defines a render(parent, nextSibling) method which TiddlyWiki calls in response to various events such as changes in the model.
(The potential inefficiency of this approach is mitigated via a refresh(changedTiddlers) method which TiddlyWiki calls to ask your widget whether its current rendering is stale.)
While TiddlyWiki's extended WikiText is similar in design to HTML templating languages with logic constructs, you can't just ignore it and assemble all of your content in raw Javascript because it is used to define most of TiddlyWiki's UI.
In this respect, it's closer to a glue language like Qt Quick or Python with Javascript filling the "create new components" role of C/C++ in widget toolkits like Qt and GTK+.
To familiarise yourself with this, read Widgets in WikiText and Introduction to Filters. then examine the internals for a tiddler like TaskManagementExample.
TiddlyWiki builds on some basic concepts. First, TiddlyWiki should not be perceived as a dynamic web page like traditional server-side generated web pages. Instead TiddlyWiki can be perceived as an application which is written entirely in JavaScript and uses HTML5 and CSS3 to render a GUI. This way TiddlyWiki can be executed in any JavaScript environment like a browser or a node.js instance, while keeping the advantages of a simple application. One of these advantages is, that TiddlyWiki has no need to emulate an application state, as a traditional web application would need to do.
A second idea concerns the storage of the application data. In contrast to a traditional web application, TiddlyWiki doesn't store the data in an external database but simply uses native data structures already existing in JavaScript to store tiddlers, the basic (atomic) element of the TiddlyWiki application, in the memory. Additional core modules provide a way to persist this storage in simple HTML div elements.
Just by building on these simple and basic concepts,
These points already enable TW to be used as an offline-enabled single file web application. Also, by using a server side node.js environment running the same TiddlyWiki application, TiddlyWiki can be used as an online web application. This is realized on server-side by providing an additional module to persist tiddlers into plain text files and on client-side by a module syncing the local data store with the node.js server.
TiddlyWiki is a large project with many interested parties. It benefits everyone if the code is as easy to read as possible. A key part of that it must be written and laid out consistently – the code should read as though it were written by a single author.
TiddlyWiki uses 4-character tabs for indenting.
One blank line is used to separate blocks of code. Occasional blank lines are permitted within blocks for clarity, but should be avoided unless they solve a specific readability problem.
See the following example for layout of basic JavaScript constructs:
/*
Multiline comments are used to introduce a block of code such as a function definition
*/
function demoFunction(param,more) {
// Proper sentence capitalisation for comments
if(condition == "something") {
// No space between "if" and the brackets; always spaces around binary operators
something = somethingElse;
myOtherFunction(one,two); // No whitespace within function parameters
do {
myCondition.explore(); // Always use semicolons
} while(myCondition < worsens);
}
}
Double quotes are preferred over single quotes for string literals.
The microkernel builds up the base functionality to manage tiddlers by providing a basic wiki store and a barebone tiddler model. The microkernel can also load a set of (decrypted) tiddlers and provides a module system, enabling a developer to extend the kernel and add functionality with module tiddlers and plug-ins.
For instance, the TiddlyWiki Core Application is provided as a single plug-in. In this part we want to focus on the TiddlyWiki core plug-in. After describing how new functionality is added directly to the wiki store, we show how the core plug-in realises persistence of tiddlers. The last part describes how TiddlyWiki builds an UI out of tiddlers and WikiText.
The TiddlyWiki Application consists of a microkernel and several modules building up the full application.
Modules with module-type: startup have to export a function named startup and may export a name property to identify this module. The startup function will be executed by the boot kernel at the end of the boot process.
// From boot.js:
// Gather up any startup modules
$tw.boot.remainingStartupModules = []; // Array of startup modules
$tw.modules.forEachModuleOfType("startup",function(title,module) {
if(module.startup) {
$tw.boot.remainingStartupModules.push(module);
}
});
// Keep track of the startup tasks that have been executed
$tw.boot.executedStartupModules = Object.create(null);
$tw.boot.disabledStartupModules = $tw.boot.disabledStartupModules || [];
// Repeatedly execute the next eligible task
$tw.boot.executeNextStartupTask();executeNextStartupTask() will execute the remaining startup modules in remainingStartupModules. A startup module can export the variables before and/or after, each containing an array of names of other startup modules. executeNextStartupTask() will use this information to execute the modules in the correct order.
Startup modules can be marked as synchronous by exporting synchronous = true. If synchronous is set to false, the startup function is executed with an callback function as the first argument. The startup function has to call this function to allow subsequent startup modules to get executed. This is necessary when the startup function itself uses asynchronous calls.
Most of the following mechanisms need a way to get notified, when anything in the wiki store changes. The core -plug-in adds an event system to the bare wiki store. The event system provides the ability to listen to events. The most important is the "change" event which notifies the listeners when tiddlers have changed, with a list of the changed tiddlers. The event mechanism is one of the few mechanisms which needs a hook at the microkernel: The microkernel contains an empty function "enqueueTiddlerEvent(event)" and calls this function when a tiddler is added or deleted. The event mechanism from the core plug-in overwrites this function with it's own implementation. The functions providing the event system are added via the wikimethod module type.
The core plug-in adds two caches to the wiki store. A global cache and a cache for each tiddler. While the global cache is cleared whenever any tiddler changes, the tiddler cache is only cleared when the associated tiddler changes. The idea of the tiddler cache is, that sometimes we want to calculate information from a tiddler but don't want to recalculate every time we need this information. Imagine a tiddler which contains links to other tiddlers and at one point we want to have a list of the linked tiddlers (LinksFilter). With the tiddler cache we can parse the WikiText, extract the linked tiddlers and put this list in the cache. Until the tiddler is changed, the cache can be used to access this information in the next requests.
The same idea holds for the global cache. For example when a filter string is parsed and a list of matching tiddlers is constructed. This list can be put in the global cache for later reuse until something changes in the store.
Like the Event Mechanism, the cache needs hook functions in the microkernel.
The microkernel calls clearGlobalCache() and clearCache(tiddlertitle) when a tiddler changes.
The core's cache mechanism overwrites this functions.
The functions providing the cache system are added via the wikimethod module type.
The core plug-in introduces a segregation of tiddlers. The tiddler model is central in the whole TiddlyWiki application and can be used in various roles. Because of the fact that a TiddlyWiki user works with tiddlers (taking the role of a wiki page) and tiddlers are the building blocks of the whole application (including modules, UI elements, etc.) we need to identify the internal tiddlers.
The core plug-in introduces the concept of system tiddlers. It builds on the convention that application internal tiddler names start with $:/.
Then the core plug-in introduces a set of new functions to the wiki store which are used to retrieve tiddlers like getTiddlers(options) and forEachTiddler(callback, options).
These functions work with all tiddlers in the store but the options parameter provides the ability to sort tiddlers by a field-name and exclude tiddlers with a specific tag.
By default it doesn't return system tiddlers. To get a list of all tiddlers including system tiddlers, this must be requested explicitly via the options.
If a function wants to present a list of tiddlers to the user it can use this new functions so that internal application tiddlers wouldn't clutter the resulting list.
These functions are added via the wikimethod module type.
The core plug-in extends the store by a simple mechanism to tag a tiddler. This provides the functionality to
{ tag: [tiddler1, tiddler3, tiddler3] }The tags are stored directly in the tiddler. Each tiddler can have a field named "tag", which contains an array of its tags. The above functions use this field to calculate their results and cache them in a global cache.
Organising information with tags is easy, intuitive and is common throughout the web. In most cases the functions mentioned above are not enough and a more sophisticated way of querying is needed. But instead of focusing only on tags TiddlyWiki introduces a querying system that isn't bound to tags or single tiddlers.
This filter mechanism is build on the idea of a pipeline of single filter operators. Filters are noted as strings and can look like
[tag[task]!tag[done]interesting[very]]This example would (implicitly) put all available tiddlers into the pipe.
The first operator tag[task] would only pass tiddlers which are tagged with "task".
The !tag[done] operator is negated and only passes tiddlers which are not tagged with "done".
The last filter operator passes only tiddlers with a field "interesting" set to "very".
So as a result this filter would be used to obtain all tiddlers which are marked as task, aren't already done and are very interesting.
There are many filter operators already build into the core plug-in including the mentioned tag- and field operators, filter operators to sort the tiddlerlist by a specified field, etc.
But more sophisticated operators are possible, too.
An example would be the search-operator. This filter operator looks for the searched string in the text and in the tags of the tiddlers.
If the provided filter operators are not enough, a developer can add new filters by adding a module with the filteroperator type.
Tags and the filter mechanism are used throughout the core plug-in for internal puproses.
Tags which start with $:/ are normally hidden from the casual user, similar to System Tiddlers
The filter mechanism is added to the wiki store with the wikimethod module type.
The microkernel only contains a bare store and some deserializers to load tiddlers from JSON files or from the DOM of the current HTML file. The core plug-in adds some more deserializers and a new mechanism for persisting and synchronising tiddlers.
This mechanism is provided as a global module in $:/core/modules/syncer.js. The saver module has three responsibilities:
The syncer module is connected mainly to two other modules.
For one it registers to changes at the wiki store (Event Mechanism) and if any changes occur they are synced to the remote store.
Then it provides a function saveWiki(options). This function can be used by other modules. For example the RootWidget uses this function to save the whole wiki or start downloading single tiddlers.
The syncer itself does not provide a concrete implementation of saving, downloading or syncing the tiddlers.
Instead it loads modules of type saver and syncadaptor and manages the saving/syncing process.
Modules with module-type: tiddlerdeserializer can provide functions to create tiddlers out of any textual representation. Each function must be associated with a type like application/json or text/html.
They get the textual representation of the tiddlers, some default field values and the type of the text block as arguments. They return an array of JavaScript objects representing the resulting tiddlers.
Deserializers are not managed by the syncer module. Instead the concept of deserializers is in fact part of the microkernel. This is necessary because the microkernel needs a way of loading tiddlers before it can load the core plug-in and execute it's startup modules. (Due to the fact that the core plug-in and it's modules are represented as tiddlers.)
The load-modules startup module loads additional deserializers and pushes them into the store.
The core plug-in for example contains a few deserializers which can read a whole TiddlyWiki 5 or classic HTML file and load the individual tiddlers into the store.
Modules with module-type: saver provide functionality to save the whole wiki. There are three methods a saver can support:
A saver module has to export two functions. canSave(wiki) returning true if this module is capable of working and create(wiki} returning an instance of a saver object.
This saver object has to provide an info property containing a name, a priority, an array of methods it supports and a method save(text,method,callback). This method is called from TW with the actual text which should be saved, the method which is used and a callback function to report errors: callback("Error while saving") or to notify that saving went well: callback("Saving went well :)"). If the saver method successfully saved the file it has to return true, or false otherwise.
Saves are triggered by messages from the UI. The syncer module uses the saver with the highest priority capable of the requested method to save the file.
The core plug-in contains a saver capable of saving the current state of the wiki to the local hard drive by using a special Firefox extension called Tiddlyfox. If this extension is not available, the savers canSave method would return false. A saver with a lower priority would then ask the user to save the current state as a new HTML file.
A module with module-type: syncadaptor provides functionality to get a list of tiddlers (this list is provided as SkinnyTiddlers, which are normal tiddlers without the text field) and to load, save and delete single tiddlers. A syncadaptor can also provide functions to login and logout so that syncadaptor modules can be used to synchronize tiddlers with a remote server.
The syncer module only uses one syncadaptor and honours a special system tiddler $:/config/SyncFilter containing a filter string. Tiddlers matching this filter string are not synced with a syncadapter.
The microkernel provides basic functionality to store tiddlers and manage modules and plugins. The Startup Process then loads the core plug-in including extensions to the store providing a Event Mechanism, Caching, Tags and a Filter Mechanism giving the ability to query for specific tiddlers.
Using some of this techniques the core plug-in also adds some functionalities to load and save single tiddlers or the whole wiki, described in Extended Persistence.
This next chapter will focus on the parts of the core plug-in that provide the UI of TiddlyWiki.
The rendering pipeline [http://tiddlywiki.com/talkytalky, 17.07.2014]
The first stage of WikiText processing is the parser.
A Parser is provided by a module with module-type: parser and is responsible to transform block of text to a parse-tree.
The parse-tree consists of nested nodes like
{type: "element", tag: <string>, attributes: {}, children: []} - an HTML element
{type: "text", text: <string>} - a text node
{type: "entity", value: <string>} - an HTML entity like © for a copyright symbol
{type: "raw", html: <string>} - raw HTMLThe core plug-in provides a recursive descent WikiText parser which loads it's individual rules from individual modules.
Thus a developer can provide additional rules by using module-type: wikirule. Each rule can produce a list of parse-tree nodes.
A simple example for a wikirule producing a <hr> from --- can be found in horizrule.js
HTML tags can be embedded into WikiText because of the html rule.
This rule matches HTML tag syntax and creates type: "element" nodes.
But the html-rule has another special purpose. By parsing the HTML tag syntax it implicitly parses WikiText widgets.
It the recognises them by the $ character at the beginning of the tag name and instead of producing "element" nodes
it uses the tag name for the type:
{type: "list", tag: "$list", attributes: {}, children: []} - a list elementThe Widgets part will reveal why this makes sense and how each node is transformed into a widget. Another special characteristic of the html-rule or the parse nodes in general is the attributes property. Attributes in the parse-tree are not stored as simple strings but they are nodes of its own to make indirect text references available as attributes as described in Widgets:
{type: "string", value: <string>} - literal string
{type: "indirect", textReference: <textReference>} - indirect through a text referenceWhen the WikiText has been transformed into a parse-tree the next step is to transform this parse-tree into a widget-tree.
We talked about widgets as parts of the WikiText markup but in fact each node of the parse-tree is transformed to a widget object.
The core plug-in provides a basic widget object which gets the parse node it should represent and a DOM node. The widget then must create the DOM structure which represents the parse node and add it to the provided DOM node.
A LinkWidget for example would create the DOM node for a <a>...</a> tag and put it in the provided DOM node.
When a widget gets a parse node with child nodes it must create the corresponding child widgets.
All this functionality is basically provided with the base widget. But when creating the widget for a parse node it loads additional modules with module-type: widget. These modules can export multiple widgets which extend the base widget and can override it's methods like the rendering and refresh functions.
As described in Parser each parse node contains a "type" property. This type directly determines which widget module is used.
{type: "text", text: <string>} - a text nodeWould be transformed to a TextWidget. (Note that the TextWidget module exports a property "text".)
So in fact when talking about widgets in WikiText, they are not a feature added to WikiText but the ability to directly use a specific widget instead of the parser choosing a widget type.
In the beginning we talked about widgets and how they enable dynamic behaviour. But up until now we only described how widgets create DOM nodes from parse nodes. Widgets add dynamic behaviour in two ways.
Messages are events that are triggered by the user. They are generated by widgets for example when the user clicks on a ButtonWidget. Each message has a type property like "tm-delete-tiddler" and a parameter.
{type: "tm-delete-tiddler", param: "MyOldTiddler"}When such a message is created by a widget it sends the message to it's parent widget which sends it to it's own parent widget and so on.
On this way each widget can try to dispatch the message.
This concept is realised in the base widget object.
It provides a function dispatchEvent(message) which is called by the children to bubble the message up the widget tree.
Another function addEventListener(type,listener) can be used to bind a function to a specific message type.
If the listener returns false, the message is send to the parent widget.
The TiddlyWiki core plug-in handles a lot of messages in the NavigatorWidget.
With Messages a widget is able to put some kind of events into the TiddlyWiki application which is one part of the dynamic behaviour of widgets.
The other part is selective updating.
Widgets are often dependant on tiddler states.
The ListWidget for example can be configured to list all tiddlers which are tagged with "important".
Now, when such a tiddler is changed and it's "important" tag is removed, this change should reflect in the ListWidget.
To allow widgets to react on such changes, each widget can provide a function refresh(changedTiddlers).
The RootWidget is registered to the wiki store, using the Event Mechanism. When an change event occurs it starts to call the refresh function of its children with a list of the changed tiddlers.
Each widget can then decide if it has to change or re-render its DOM representation and call the refresh function of its own children.
This way every time a tiddler or the wiki store itself changes, each widget can instantly react on these changes or ignore them.
Another way of updating are text reference attributes (text references explained in Transclusion and TextReference):
{type: "indirect", textReference: <textReference>}When a widget got a attribute which is a text reference and the refresh function is called, it can check if the text reference references a changed tiddler and update accordingly.
Nearly every state of the UI is stored in tiddlers. A search mechanism for example would use a EditTextWidget which shows an text input field for the search string and a ListWidget to show the results. The EditTextWidget is bound to a tiddler $:/temp/search. Meaning when editing the text in the input field the tiddlers content is changed to this text. On the other hand when the tiddler changes the RootWidget is notified. It then starts calling its childrens refresh methods. Eventually the refresh method of the EditTextWidget is called and it can re-render itself if necessary. This way TiddlyWiki can re-use an already existing data structure which is not only convenient because we don't need to introduce an additional structure but tiddlers managed in the wiki store are already a pretty powerful data structure, supporting an Event Mechanism (so we don't need an additional observer pattern), Caching, Metadata by additional tiddler fields and a way to obtain specific tiddlers.
The previous parts about Widgets and the Parser explained how a block of WikiText is transformed into a DOM representation and how this presentation can react on changes to the wiki store. The previous chapters also describe that WikiText is saved in individual tiddlers, including the WikiText describing the UI components. This raises the question, how these multiple tiddlers are build up to a single UI. But before answering this question we need to introduce text references and transclusion.
A text reference describes a special notation to indirectly refer to the contents of a specified tiddler field. The syntax of a text reference is:
<tiddlertitle>
<tiddlertitle>!!<fieldname>
!!<fieldname> - specifies a field of the current tiddlersTo obtain the actual text, the core plug-in adds a function to the wiki store getTextReference(textRef,defaultText,currTiddlerTitle). The "currentTiddlerTitle" is the title of a tiddler which is used when no tiddlerTitle is specified in the text reference.
What the currentTiddler should be, depends on where the text reference is used.
If it is for example used as a widget attribute, the current tiddler is the tiddler which contains this widget.
Text references are used by widgets (attributes), filteroperators (parameter) and in transclusions.
These elements use the getTextReference(textRef,defaultText,currTiddlerTitle) function.
Transclusion means including the contents of a different tiddler. This is realized with a transclude widget which parses the tiddler to be included and adds the resulting nodes as children of its own parse node. The trick with transclusion is, that it shows the content of a different tiddler but by default it does not change the current tiddler. This enables us to create tiddlers with text references and use them as a templates.
For example:
Tiddler MyTask having the fields and the content of:
important: "very"
assoc.person: "Hans Dampf"
<$transclude tiddler="TaskHeaderTemplate" />
Hans needs some more Dampf.And Tiddler TaskHeaderTemplate with a content of:
<$view field="assoc.person"/> has a <$view field="important"/> important task for us:When showing tiddler MyTask it would result in:
Hans Dampf has a very important task for us:
Hans needs some more Dampf.Transclusion and templates is one of the most important concepts in TiddlyWiki. It allows us to include other tiddlers using the metadata for our current tiddler. It also allows us to transclude a template tiddler with a third tiddler set as the currentTiddler with the TiddlerWidget:
<$tiddler tiddler="MyTiddler">
<$transclude tiddler="EditTemplate" />
</$tiddler>This way we can create a different view on a tiddler which does not only show it's title and it's content but shows it's content and metadata in editable text fields and allows us to edit this tiddler. Also the template concept is used by the ListWidget to show the tiddlers through a specified template. Finally, when wanting to download the wiki as a new html file, this html file is created by binding a list of all tiddlers to a template and sending the resulting text to the syncer module described in Extended Persistence to save the current wiki.
The previous parts of this chapter showed how WikiText is transformed to DOM nodes which dynamically react to tiddler changes and a way to compose tiddlers from other tiddlers. This last part describes how the TiddlyWiki core plug-in starts up a UI build from tiddlers and WikiText.
After the microkernel has loaded it starts executing Startup Modules. The core plug-in contains two startup modules which are responsible to kick off the UI:
rootwidget.js is a startup module and creates an instance of the base widget.
This widget is globally accessible $tw.rootWidget.
The DOM node associated to this widget is the current browser window's DOM (document).
At first, the root widget has no children but provides some basic event handlers (Messages) like:
After the root widget is loaded another startup module $:/core/modules/startup/render.js creates a transclude widget which contains the contents of $:/core/ui/PageTemplate which is now bound to the browsers DOM document. The render function of the transclude widget is initially executed and a listener is registered at the store which executes the refresh function of the transclude widget to trigger the Selective Update process.
Techniques for including other tiddlers and Templates are finally used in $:/core/ui/PageTemplate to build the TiddlyWiki UI only from tiddlers written in WikiText (with widgets implemented in javascript):
For example to implement the list of open wiki pages the $:/core/ui/PageTemplate contains a navigator widget which maintains a list of open tiddlers in a field of $:/StoryList and handles events like tm-navigate by adding a tiddler specified as parameter to the top of the list in $:/StoryList.
The story tiddler transcluded in $:/core/ui/PageTemplate then uses a ListWidget to transclude all tiddlers in $:/StoryList through a special template $:/core/ui/ViewTemplate.
A event of the type tm-close-tiddler would remove a specified tiddler from $:/StoryList.
The Event Mechanism would trigger a changed event which triggers a call of the ListWidget's refresh function which would remove the tiddler from the list, closing the tiddler.
TiddlyWiki is published as OpenSource which means that anyone can read the code and contribute to its development.
If you're interested in understanding more about the internal operation of TiddlyWiki, TiddlyWiki Architecture gives an overview of how TiddlyWiki is structured. Then read the code – start with the boot kernel $:/boot/boot.js.
TiddlyWiki's architecture is very different from an HTML page written using jQuery. This section concisely explains what TiddlyWiki does differently. It may not make much sense on the first reading.
The key to understanding how it works internally is to see that the RefreshMechanism requires that any region of the DOM can be regenerated at any time. This means that the entire state of the user interface must reside in the tiddler store, from where it is synchronised into the DOM. This is done to improve performance by minimising the DOM interactions during the refresh cycles.
It also determines the standard UI flow:
From a technical perspective, TiddlyWiki is a fairly classic MVC architecture, with strict separation of concerns. The model is the tiddler store, the view is a rendering tree (such as the one created from $:/core/ui/PageTemplate in startup.js), and the controller is the core code itself.
node-webkit allows TiddlyWiki to be set up as a native application for Windows, Mac OS X or Linux.
index.htmlpackage.json with the following content:MP3 audio and H264 video are not supported without special steps described on the node-webkit wiki.
This information is for people who are working on the development of TiddlyWiki5 itself, and isn't relevant for end users
Installing TiddlyWiki5 with NPM downloads a snapshot release of TiddlyWIki5. To use a development copy of the TiddlyWiki5 repository instead of the copy installed by NPM, use this command within the root of the TiddlyWiki5 repo:
npm linkAs releases are made during development it is necessary to adjust the version number of the TiddlyWiki5 core. This is done with the npm version command. For example:
npm version 5.0.0-alpha.10As described in #10 in this article by npm's author, when run from within a git repo this command will also commit the change and tag it
This section shows a quick and short overview over the startup process of TW, from the first step of the boot mechanism until the loading of the different startup modules. The image shown below shall point out the main parts of this startup-process.
The previous parts about Widgets and the Parser explained how a block of WikiText is transformed into a DOM representation and how this presentation can react on changes to the wiki store. The previous chapters also describe that WikiText is saved in individual tiddlers, including the WikiText describing the UI components. This raises the question, how these multiple tiddlers are build up to a single UI. But before answering this question we need to introduce text references and transclusion.
A text reference describes a special notation to indirectly refer to the contents of a specified tiddler field. The syntax of a text reference is:
<tiddlertitle>
<tiddlertitle>!!<fieldname>
!!<fieldname> - specifies a field of the current tiddlersTo obtain the actual text, the core plug-in adds a function to the wiki store getTextReference(textRef,defaultText,currTiddlerTitle). The "currentTiddlerTitle" is the title of a tiddler which is used when no tiddlerTitle is specified in the text reference.
What the currentTiddler should be, depends on where the text reference is used.
If it is for example used as a widget attribute, the current tiddler is the tiddler which contains this widget.
Text references are used by widgets (attributes), filteroperators (parameter) and in transclusions.
These elements use the getTextReference(textRef,defaultText,currTiddlerTitle) function.
Transclusion means including the contents of a different tiddler. This is realized with a transclude widget which parses the tiddler to be included and adds the resulting nodes as children of its own parse node. The trick with transclusion is, that it shows the content of a different tiddler but by default it does not change the current tiddler. This enables us to create tiddlers with text references and use them as a templates.
For example:
Tiddler MyTask having the fields and the content of:
important: "very"
assoc.person: "Hans Dampf"
<$transclude tiddler="TaskHeaderTemplate" />
Hans needs some more Dampf.And Tiddler TaskHeaderTemplate with a content of:
<$view field="assoc.person"/> has a <$view field="important"/> important task for us:When showing tiddler MyTask it would result in:
Hans Dampf has a very important task for us:
Hans needs some more Dampf.Transclusion and templates is one of the most important concepts in TiddlyWiki. It allows us to include other tiddlers using the metadata for our current tiddler. It also allows us to transclude a template tiddler with a third tiddler set as the currentTiddler with the TiddlerWidget:
<$tiddler tiddler="MyTiddler">
<$transclude tiddler="EditTemplate" />
</$tiddler>This way we can create a different view on a tiddler which does not only show it's title and it's content but shows it's content and metadata in editable text fields and allows us to edit this tiddler. Also the template concept is used by the ListWidget to show the tiddlers through a specified template. Finally, when wanting to download the wiki as a new html file, this html file is created by binding a list of all tiddlers to a template and sending the resulting text to the syncer module described in Extended Persistence to save the current wiki.
The microkernel provides basic functionality to store tiddlers and manage modules and plugins. The Startup Process then loads the core plug-in including extensions to the store providing a Event Mechanism, Caching, Tags and a Filter Mechanism giving the ability to query for specific tiddlers.
Using some of this techniques the core plug-in also adds some functionalities to load and save single tiddlers or the whole wiki, described in Extended Persistence.
This next chapter will focus on the parts of the core plug-in that provide the UI of TiddlyWiki.
The rendering pipeline [http://tiddlywiki.com/talkytalky, 17.07.2014]
The first stage of WikiText processing is the parser.
A Parser is provided by a module with module-type: parser and is responsible to transform block of text to a parse-tree.
The parse-tree consists of nested nodes like
{type: "element", tag: <string>, attributes: {}, children: []} - an HTML element
{type: "text", text: <string>} - a text node
{type: "entity", value: <string>} - an HTML entity like © for a copyright symbol
{type: "raw", html: <string>} - raw HTMLThe core plug-in provides a recursive descent WikiText parser which loads it's individual rules from individual modules.
Thus a developer can provide additional rules by using module-type: wikirule. Each rule can produce a list of parse-tree nodes.
A simple example for a wikirule producing a <hr> from --- can be found in horizrule.js
HTML tags can be embedded into WikiText because of the html rule.
This rule matches HTML tag syntax and creates type: "element" nodes.
But the html-rule has another special purpose. By parsing the HTML tag syntax it implicitly parses WikiText widgets.
It the recognises them by the $ character at the beginning of the tag name and instead of producing "element" nodes
it uses the tag name for the type:
{type: "list", tag: "$list", attributes: {}, children: []} - a list elementThe Widgets part will reveal why this makes sense and how each node is transformed into a widget. Another special characteristic of the html-rule or the parse nodes in general is the attributes property. Attributes in the parse-tree are not stored as simple strings but they are nodes of its own to make indirect text references available as attributes as described in Widgets:
{type: "string", value: <string>} - literal string
{type: "indirect", textReference: <textReference>} - indirect through a text referenceWhen the WikiText has been transformed into a parse-tree the next step is to transform this parse-tree into a widget-tree.
We talked about widgets as parts of the WikiText markup but in fact each node of the parse-tree is transformed to a widget object.
The core plug-in provides a basic widget object which gets the parse node it should represent and a DOM node. The widget then must create the DOM structure which represents the parse node and add it to the provided DOM node.
A LinkWidget for example would create the DOM node for a <a>...</a> tag and put it in the provided DOM node.
When a widget gets a parse node with child nodes it must create the corresponding child widgets.
All this functionality is basically provided with the base widget. But when creating the widget for a parse node it loads additional modules with module-type: widget. These modules can export multiple widgets which extend the base widget and can override it's methods like the rendering and refresh functions.
As described in Parser each parse node contains a "type" property. This type directly determines which widget module is used.
{type: "text", text: <string>} - a text nodeWould be transformed to a TextWidget. (Note that the TextWidget module exports a property "text".)
So in fact when talking about widgets in WikiText, they are not a feature added to WikiText but the ability to directly use a specific widget instead of the parser choosing a widget type.
In the beginning we talked about widgets and how they enable dynamic behaviour. But up until now we only described how widgets create DOM nodes from parse nodes. Widgets add dynamic behaviour in two ways.
Messages are events that are triggered by the user. They are generated by widgets for example when the user clicks on a ButtonWidget. Each message has a type property like "tm-delete-tiddler" and a parameter.
{type: "tm-delete-tiddler", param: "MyOldTiddler"}When such a message is created by a widget it sends the message to it's parent widget which sends it to it's own parent widget and so on.
On this way each widget can try to dispatch the message.
This concept is realised in the base widget object.
It provides a function dispatchEvent(message) which is called by the children to bubble the message up the widget tree.
Another function addEventListener(type,listener) can be used to bind a function to a specific message type.
If the listener returns false, the message is send to the parent widget.
The TiddlyWiki core plug-in handles a lot of messages in the NavigatorWidget.
With Messages a widget is able to put some kind of events into the TiddlyWiki application which is one part of the dynamic behaviour of widgets.
The other part is selective updating.
Widgets are often dependant on tiddler states.
The ListWidget for example can be configured to list all tiddlers which are tagged with "important".
Now, when such a tiddler is changed and it's "important" tag is removed, this change should reflect in the ListWidget.
To allow widgets to react on such changes, each widget can provide a function refresh(changedTiddlers).
The RootWidget is registered to the wiki store, using the Event Mechanism. When an change event occurs it starts to call the refresh function of its children with a list of the changed tiddlers.
Each widget can then decide if it has to change or re-render its DOM representation and call the refresh function of its own children.
This way every time a tiddler or the wiki store itself changes, each widget can instantly react on these changes or ignore them.
Another way of updating are text reference attributes (text references explained in Transclusion and TextReference):
{type: "indirect", textReference: <textReference>}When a widget got a attribute which is a text reference and the refresh function is called, it can check if the text reference references a changed tiddler and update accordingly.
Nearly every state of the UI is stored in tiddlers. A search mechanism for example would use a EditTextWidget which shows an text input field for the search string and a ListWidget to show the results. The EditTextWidget is bound to a tiddler $:/temp/search. Meaning when editing the text in the input field the tiddlers content is changed to this text. On the other hand when the tiddler changes the RootWidget is notified. It then starts calling its childrens refresh methods. Eventually the refresh method of the EditTextWidget is called and it can re-render itself if necessary. This way TiddlyWiki can re-use an already existing data structure which is not only convenient because we don't need to introduce an additional structure but tiddlers managed in the wiki store are already a pretty powerful data structure, supporting an Event Mechanism (so we don't need an additional observer pattern), Caching, Metadata by additional tiddler fields and a way to obtain specific tiddlers.
The previous parts about Widgets and the Parser explained how a block of WikiText is transformed into a DOM representation and how this presentation can react on changes to the wiki store. The previous chapters also describe that WikiText is saved in individual tiddlers, including the WikiText describing the UI components. This raises the question, how these multiple tiddlers are build up to a single UI. But before answering this question we need to introduce text references and transclusion.
A text reference describes a special notation to indirectly refer to the contents of a specified tiddler field. The syntax of a text reference is:
<tiddlertitle>
<tiddlertitle>!!<fieldname>
!!<fieldname> - specifies a field of the current tiddlersTo obtain the actual text, the core plug-in adds a function to the wiki store getTextReference(textRef,defaultText,currTiddlerTitle). The "currentTiddlerTitle" is the title of a tiddler which is used when no tiddlerTitle is specified in the text reference.
What the currentTiddler should be, depends on where the text reference is used.
If it is for example used as a widget attribute, the current tiddler is the tiddler which contains this widget.
Text references are used by widgets (attributes), filteroperators (parameter) and in transclusions.
These elements use the getTextReference(textRef,defaultText,currTiddlerTitle) function.
Transclusion means including the contents of a different tiddler. This is realized with a transclude widget which parses the tiddler to be included and adds the resulting nodes as children of its own parse node. The trick with transclusion is, that it shows the content of a different tiddler but by default it does not change the current tiddler. This enables us to create tiddlers with text references and use them as a templates.
For example:
Tiddler MyTask having the fields and the content of:
important: "very"
assoc.person: "Hans Dampf"
<$transclude tiddler="TaskHeaderTemplate" />
Hans needs some more Dampf.And Tiddler TaskHeaderTemplate with a content of:
<$view field="assoc.person"/> has a <$view field="important"/> important task for us:When showing tiddler MyTask it would result in:
Hans Dampf has a very important task for us:
Hans needs some more Dampf.Transclusion and templates is one of the most important concepts in TiddlyWiki. It allows us to include other tiddlers using the metadata for our current tiddler. It also allows us to transclude a template tiddler with a third tiddler set as the currentTiddler with the TiddlerWidget:
<$tiddler tiddler="MyTiddler">
<$transclude tiddler="EditTemplate" />
</$tiddler>This way we can create a different view on a tiddler which does not only show it's title and it's content but shows it's content and metadata in editable text fields and allows us to edit this tiddler. Also the template concept is used by the ListWidget to show the tiddlers through a specified template. Finally, when wanting to download the wiki as a new html file, this html file is created by binding a list of all tiddlers to a template and sending the resulting text to the syncer module described in Extended Persistence to save the current wiki.
The previous parts of this chapter showed how WikiText is transformed to DOM nodes which dynamically react to tiddler changes and a way to compose tiddlers from other tiddlers. This last part describes how the TiddlyWiki core plug-in starts up a UI build from tiddlers and WikiText.
After the microkernel has loaded it starts executing Startup Modules. The core plug-in contains two startup modules which are responsible to kick off the UI:
rootwidget.js is a startup module and creates an instance of the base widget.
This widget is globally accessible $tw.rootWidget.
The DOM node associated to this widget is the current browser window's DOM (document).
At first, the root widget has no children but provides some basic event handlers (Messages) like:
After the root widget is loaded another startup module $:/core/modules/startup/render.js creates a transclude widget which contains the contents of $:/core/ui/PageTemplate which is now bound to the browsers DOM document. The render function of the transclude widget is initially executed and a listener is registered at the store which executes the refresh function of the transclude widget to trigger the Selective Update process.
Techniques for including other tiddlers and Templates are finally used in $:/core/ui/PageTemplate to build the TiddlyWiki UI only from tiddlers written in WikiText (with widgets implemented in javascript):
For example to implement the list of open wiki pages the $:/core/ui/PageTemplate contains a navigator widget which maintains a list of open tiddlers in a field of $:/StoryList and handles events like tm-navigate by adding a tiddler specified as parameter to the top of the list in $:/StoryList.
The story tiddler transcluded in $:/core/ui/PageTemplate then uses a ListWidget to transclude all tiddlers in $:/StoryList through a special template $:/core/ui/ViewTemplate.
A event of the type tm-close-tiddler would remove a specified tiddler from $:/StoryList.
The Event Mechanism would trigger a changed event which triggers a call of the ListWidget's refresh function which would remove the tiddler from the list, closing the tiddler.
TW is built up from the micro kernel and uses the module mechanism to provide various ways of loading and saving tiddlers, including the ability to load and save to a single HTML file. Furthermore a developer can extend the application by providing modules with a specific module-type. TW searches for modules with these specific module-types and handles them accordingly.
The last sequence of the boot kernel is to execute startup modules. One of these startup modules ("load-modules") is responsible for registering some modules with specific module types at the right place. For example, the methods exported by wikimethod modules are put in \textit{\$tw.Wiki.prototype}. Other startup modules build up the initial UI and link events to certain modules.
Widget modules are used as part of the RenderingMechanism to implement each type of renderable entity. As well as the widgets that are familiar to end users, the following primitives are also implemented as widgets:
All widgets inherit from a base widget class that is defined in $:/core/modules/widgets/widget.js.
The following widget properties are defined by the core. The lifecycle of a widget object is largely a matter of maintaining the consistency of these internal properties in the face of external state changes. Individual widgets usually add their own additional properties too.
| Name | Description |
|---|---|
| parseTreeNode | Reference to the parse tree node corresponding to this widget |
| wiki | Reference to the Wiki object associated with this widget |
| variables | Hashmap of information about each widget variable (see below) |
| parentWidget | Reference to the parent widget |
| document | Reference to the document object associated with this widget. Usually either the browser global document variable or a reference to the FakeDomMechanism's $tw.fakeDocument |
| attributes | Hashmap of information about each attribute attached to this widget (see below) |
| children | Array of child widgets |
| domNodes | For widgets that directly generate DOM nodes, an array of the generated nodes |
| eventListeners | Array of event listener definitions |
The widget variables defined on a widget are stored in a hashmap of the variable name. The hashmap contains:
name: name of variableparams: array of parameters for macro definitions, each {name: "<name>", default: "<optionaldefault>"}value: string value of variableThe widget attributes associated with a widget are stored in a hashmap of the attribute name. The hashmap contains an object that describes the attribute value. Currently three attribute value types are supported:
{type: "string", value: "<value>"}{type: "indirect", textReference: "<textref>"}{type: "macro", value: {name: "<macroname>", params: [{name: "<paramname>", value: "<paramvalue>"}, ... ]}The event listeners attached to a widget are stored as a hashmap by event type. Each value is a handler function that accepts a single event parameter.
The individual methods defined by the widget object are documented in the source code of $:/core/modules/widgets/widget.js. Here we give an overview of the overall lifecycle, and how the methods fit together
initialise methodwidgetClasses methodrender methodexecute methodgetVariable methodsubstituteVariableParameters methodsubstituteVariableReferences methodevaluateMacroModule methodsetVariable methodhasVariable methodgetStateQualifier methodcomputeAttributes methodhasAttribute methodgetAttribute methodassignAttributes methodmakeChildWidgets methodmakeChildWidget methodrenderChildren methodaddEventListeners methodaddEventListener methoddispatchEvent methodrefresh methodrefreshSelf methodrefreshChildren methodfindNextSiblingDomNode methodfindFirstDomNode methodremoveChildDomNodes methodWhen the WikiText has been transformed into a parse-tree the next step is to transform this parse-tree into a widget-tree.
We talked about widgets as parts of the WikiText markup but in fact each node of the parse-tree is transformed to a widget object.
The core plug-in provides a basic widget object which gets the parse node it should represent and a DOM node. The widget then must create the DOM structure which represents the parse node and add it to the provided DOM node.
A LinkWidget for example would create the DOM node for a <a>...</a> tag and put it in the provided DOM node.
When a widget gets a parse node with child nodes it must create the corresponding child widgets.
All this functionality is basically provided with the base widget. But when creating the widget for a parse node it loads additional modules with module-type: widget. These modules can export multiple widgets which extend the base widget and can override it's methods like the rendering and refresh functions.
As described in Parser each parse node contains a "type" property. This type directly determines which widget module is used.
{type: "text", text: <string>} - a text nodeWould be transformed to a TextWidget. (Note that the TextWidget module exports a property "text".)
So in fact when talking about widgets in WikiText, they are not a feature added to WikiText but the ability to directly use a specific widget instead of the parser choosing a widget type.
In the beginning we talked about widgets and how they enable dynamic behaviour. But up until now we only described how widgets create DOM nodes from parse nodes. Widgets add dynamic behaviour in two ways.
Messages are events that are triggered by the user. They are generated by widgets for example when the user clicks on a ButtonWidget. Each message has a type property like "tm-delete-tiddler" and a parameter.
{type: "tm-delete-tiddler", param: "MyOldTiddler"}When such a message is created by a widget it sends the message to it's parent widget which sends it to it's own parent widget and so on.
On this way each widget can try to dispatch the message.
This concept is realised in the base widget object.
It provides a function dispatchEvent(message) which is called by the children to bubble the message up the widget tree.
Another function addEventListener(type,listener) can be used to bind a function to a specific message type.
If the listener returns false, the message is send to the parent widget.
The TiddlyWiki core plug-in handles a lot of messages in the NavigatorWidget.
With Messages a widget is able to put some kind of events into the TiddlyWiki application which is one part of the dynamic behaviour of widgets.
The other part is selective updating.
Widgets are often dependant on tiddler states.
The ListWidget for example can be configured to list all tiddlers which are tagged with "important".
Now, when such a tiddler is changed and it's "important" tag is removed, this change should reflect in the ListWidget.
To allow widgets to react on such changes, each widget can provide a function refresh(changedTiddlers).
The RootWidget is registered to the wiki store, using the Event Mechanism. When an change event occurs it starts to call the refresh function of its children with a list of the changed tiddlers.
Each widget can then decide if it has to change or re-render its DOM representation and call the refresh function of its own children.
This way every time a tiddler or the wiki store itself changes, each widget can instantly react on these changes or ignore them.
Another way of updating are text reference attributes (text references explained in Transclusion and TextReference):
{type: "indirect", textReference: <textReference>}When a widget got a attribute which is a text reference and the refresh function is called, it can check if the text reference references a changed tiddler and update accordingly.
Nearly every state of the UI is stored in tiddlers. A search mechanism for example would use a EditTextWidget which shows an text input field for the search string and a ListWidget to show the results. The EditTextWidget is bound to a tiddler $:/temp/search. Meaning when editing the text in the input field the tiddlers content is changed to this text. On the other hand when the tiddler changes the RootWidget is notified. It then starts calling its childrens refresh methods. Eventually the refresh method of the EditTextWidget is called and it can re-render itself if necessary. This way TiddlyWiki can re-use an already existing data structure which is not only convenient because we don't need to introduce an additional structure but tiddlers managed in the wiki store are already a pretty powerful data structure, supporting an Event Mechanism (so we don't need an additional observer pattern), Caching, Metadata by additional tiddler fields and a way to obtain specific tiddlers.
The startup module $:/core/modules/startup/load-modules.js in the TiddlyWiki core plug-in loads all modules of typewikimethod and puts their exported functions into the wiki store.
WikiRuleModules cover the module types wikirunrule, wikiblockrule and wikipragmarule. Modules of these types encapsulate the logic of individual parsing rules used by the WikiParser engine. For example, there is a wikirunrule module that identifies references to HTML entities by matching the pattern &<chars>;.
Pragma rules are applied at the start of a block of text, and cover definitions and declarations that affect the parsing of the rest of the text. Block rules are only applied at the beginning of a block of wikitext, while run rules can appear anywhere. The only current example of a pragma rule is for macro definitions.
Examples of block rules:
Examples of run rules:
Parser rule modules extend the $tw.WikiParserRule class. This is done by instantiating the class and then copying the exports of the rule module onto the instance. In this way, the parser rule can override the base behaviour of the $tw.WikiParserRule class. In particular, the base class incorporates logic for using regular expressions to match parse rules but this logic could be overridden by a parse rule that wanted to, say, use indexOf() instead of regular expressions.
The standard methods and properties of parser rules are as follows:
name: a string containing the name of this parse ruleinit(parser): initialisation function called immediately after the constructor with a pointer back to the parser containing this rulefindNextMatch(pos): returns the position of the next match after the specified positionparse(): parses the most recent match, returning an array of the generated parse tree nodes. Pragma rules don't return parse tree nodes but instead modify the parser object directly (for example, to add local macro definitions)The built in parser rules use regular expression matching. Such rules can take advantage of the implementation of findNextMatch() in the base $tw.WikiRule class by ensuring that their init() method creates a matchRegExp property containing the regular expression to match. The match property contains the details of the match for use in the parse() method.
The WikiText is a markup language, created especially for the requirements of the TiddlyWiki application. It is based on Markdown, but extended with some TiddlyWiki specific features. On one hand its a text-to-HTML conversion language and on the other hand its used to provide the interactive features of TiddlyWiki. The aim of this language is to allow the user of the software to focus on the writing. The WikiText is used to format Tiddlers within the TiddlyWiki application. The tags of the WikiText syntax can be used within the standard text input field. During the saving process these tags renders to HTML elements for example:
WikiText:---
Renders as:
HTML:<hr>
WikiText:[img[http://tiddlywiki.com/favicon.ico]]
Renders as: TW
HTML:<img src="http://tiddlywiki.com/favicon.ico">Furthermore the WikiText is used to access the widgets which are integrated in the application.These widgets are used to enhance the the WikiText with a rich functionality. Widgets are based on the ~HTML-Syntax but always starts with a $.
WikiText:
<$button message="tm-close-tiddler">Close Me!</$button> The WikiText is a markup language, created especially for the requirements of the TiddlyWiki application. It is based on Markdown, but extended with some TiddlyWiki specific features. On one hand its a text-to-HTML conversion language and on the other hand its used to provide the interactive features of TiddlyWiki. The aim of this language is to allow the user of the software to focus on the writing. The WikiText is used to format Tiddlers within the TiddlyWiki application. The tags of the WikiText syntax can be used within the standard text input field. During the saving process these tags renders to HTML elements for example:
WikiText:---
Renders as:
HTML:<hr>
WikiText:[img[http://tiddlywiki.com/favicon.ico]]
Renders as: TW
HTML:<img src="http://tiddlywiki.com/favicon.ico">Furthermore the WikiText is used to access the widgets which are integrated in the application.These widgets are used to enhance the the WikiText with a rich functionality. Widgets are based on the ~HTML-Syntax but always starts with a $.
WikiText:
<$button message="tm-close-tiddler">Close Me!</$button>